This paper demonstrates an experience in the development of a design performance evaluation system that can frequently evaluate building design performance within the design process in a real-time manner. The evaluation system, that we call “design compiler,” is composed of several modules such as a front-end component-based CAD engine, a knowledge base, and a set of design agents. The notion of the design compiler is quite similar to a compiler for computer programming such as a C compiler. While a computer programmer compiles a set of programming codes to check compiling errors during the implementation of a software system, an architectural designer can'compile'his/her intermediate design product to evaluate design errors during the design process. The compilation can be done immediately at any level or any time during the design process in a real-time manner. We expect that this compiling process will dramatically increase design feedbacks, and thus result in a better design product. Further research issues that have been identified at the end of the research include increasing the modelling capability, extending to multi-story building representation, developing various design agents, exploring better ways to request and manage design knowledge, and supporting design collaboration.