Binocular disparity information is an important source of 3D perception. Neurons sensitive to binocular disparity are found in almost all major visual areas in nonhuman primates. In area V4, disparity processes are suggested for the purposes of 3D-shape representation and fine disparity perception. However, whether neurons in V4 are sensitive to disparity-defined edges used in shape representation is not clear. Additionally, a functional organization for disparity edges has not been demonstrated so far. With intrinsic signal optical imaging, we studied functional organization for disparity edges in the monkey visual areas V1, V2, and V4. We found that there is an orientation map in V4 activated by edges purely defined by binocular disparity. This map is consistent with the orientation map obtained with regular luminance-defined edges, indicating a cue-invariant edge representation in this area. In contrast, such a map is much weaker in V2 and totally absent in V1. These findings reveal a hierarchical processing of 3D shape along the ventral pathway and the important role that V4 plays in shape-from-disparity detection.