Designing a building or a new urban space is collaborative work that involves several people with different backgrounds. To achieve consensus all participants in the process meet to discuss documents such as floor plans and sections. This paper reports on the progress of Space Pen, a new system to allow several users to draw on and annotate a three-dimensional representation of a building remotely over the Internet.