Understanding affordance is a first step to a deeper understanding of the world, one in which a robot knows how an object and its parts can be used. To assist in everyday activities, robots must not only be able to recognize a tool, but also localize the its parts and identify how each part is used. We propose a preliminary approach to jointly localize and identify the function, or affordances, of a tool’s parts for objects from known or completely novel categories. We combine superpixel segmentation, feature learning, and conditional random fields to provide precise 3D predictions of functional parts that can be used directly by a robot to interact with the world. To investigate this problem, we introduce a new RGB-D Part Affordance Dataset consisting of 105 kitchen, workshop, and garden tools with pixel-level affordance labels for over 10,000 RGB-D images. We analyze the effectiveness of different feature types, and show that geometric features are most important for successful affordance identification. We demonstrate that by identifying the affordances of tools at the level of parts, we can generalize to novel object categories and identify the useful parts of never before seen tools.