The　vision-based　displacement　measurement　system　was　developed,　which　using　digital　video　camcorder　to　test　the　deformation　of　high-rise　structures.　It　is　more　economical　than　contact　and　contact-less　displacement　sensors.　A　series　of　tests　were　conducted　to　investigate　the　precision,　serviceability,　and　stability　of　the　vision-based　displacement　method.　The　results　show　that,　the　proposed　method　can　effectively　test　the　dynamic　displacement,　moreover,　the　method　can　be　effectively　applied　to　test　the　displacement　caused　by　vibration　which　contains　various　frequency　components.　Based　on　the　system,　the　deformation　of　high-rise　building　structure　was　tested.　The　results　show　that,　the　displacement　obtained　by　vision-based　can　illustrate　the　free-vibration　characteristics　of　structure　well,　meanwhile,　this　method　can　test　bidirectional　displacement　in　shaking　table　test　and　practical　engineering.