Activities in the building industry in Germany concentrate increasingly on a combination of renovation and new-build. A look at current computer aided applications reveals a serious lack of IT support for the whole architectural design process with and within existing buildings (e.g. building surveying, designing on site and the preparation of information for further use in later planning stages). An ongoing interdisciplinary research project undertaken jointly by the faculty of media and the faculty of architecture is investigating methods and techniques for the computer-aided support of the design process with and within existing buildings. The goal is to develop a hardware and software concept for a “design-toolbox” based on SAR (spatial augmented reality) and to implement aspects of it as prototypes. The paper focuses on one aspect of the research project - designing with images in an AR-Environment. The paper presents an overview of the project, a description of the core model based on a client-server architecture, the historical / theoretical architectural background and first results of this research part. The project is funded by the “German Research Foundation (DFG)”.