Sketches and working models are without doubt one of the most widely used tools in the architect’s repertoire. In the early phases of the design process in particular, they represent an essential part of the exploration process. For the most part, however, sketching and model building are separate and sequential steps. This paper examines how both media can be linked to each other more effectively so that the design process is more continuous and the transition between the two media more fluid. Using a multi-touch table with additional 3D real-time object recognition, a prototypical system was developed and realised as part of a research project in which an interactive 3D sketching tool was linked with a real 3D working model.