Direct measurements of diffusion and segregation lengths are generally limited by the inherent averaging that occurs in conventional characterization techniques. We are investigating the atomic-scale dynamics of these processes using a combination of annealing and cross-sectional scanning tunneling microscopy (XSTM). To date, we have studied these issues in a number of heterostructure systems, and have made several important contributions. For example, we used the "wetting layers" between InAs/GaAs quantum dots to directly measure In-Ga interdiffusion and In segregation lengths. To our knowledge, these XSTM studies represent the first direct atom-level measurements of both of these quantities. In addition, we report the first direct evidence for anti-site vacancy diffusion, originally predicted by Van Vechten more than 30 years ago. We are in the process of developing a further understanding of anti-site vacancy diffusion through XSTM studies a variety of mixed anion compound semiconductor heterostructures.