At present, more than half of all building activity in the German building sector is undertaken within existing built contexts. The development of a conceptual and technological basis for the digital support of design directly on site, within an existing building context is the focus of the research project “Spatial Augmented Reality for Architecture” (SAR). This paper describes the goals achieved in one aspect of the project: the sampling of colors and materials at a scale of 1:1 using Augmented Reality (AR) technologies. We present initial results from the project, the development of an ad-hoc visualization of interactive data on arbitrary surfaces in real-world indoor environments using a mobile hardware setup. With this, it was possible to project the color and material qualities of a design directly onto almost all surfaces within a geometrically corrected, existing building. Initially, a software prototype “Spatial Augmented Reality for Architecture-Colored Architecture” (SAR-CA) was developed and then assessed based on evaluation results from a user study.