Background:　Web-based　social　media　provides　common　people　with　a　platform　to　express　their　emotions　conveniently　and　anonymously.　There　have　been　nearly　2　million　messages　in　a　particular　Chinese　social　media　data　source,　and　several　thousands　more　are　generated　each　day.　Therefore,　it　has　become　impossible　to　analyze　these　messages　manually.　However,　these　messages　have　been　identified　as　an　important　data　source　for　the　prevention　of　suicide　related　to　depression　disorder.　Objective:　We　proposed　in　this　paper　a　distant　supervision　approach　to　developing　a　system　that　can　automatically　identify　textual　comments　that　are　indicative　of　a　high　suicide　risk.　Methods:　To　avoid　expensive　manual　data　annotations,　we　used　a　knowledge　graph　method　to　produce　approximate　annotations　for　distant　supervision,　which　provided　a　basis　for　a　deep　learning　architecture　that　was　built　and　refined　by　interactions　with　psychology　experts.　There　were　three　annotation　levels,　as　follows:　free　annotations　(zero　cost),　easy　annotations　(by　psychology　students),　and　hard　annotations　(by　psychology　experts).　Results:　Our　system　was　evaluated　accordingly　and　showed　that　its　performance　at　each　level　was　promising.　By　combining　our　system　with　several　important　psychology　features　from　user　blogs,　we　obtained　a　precision　of　80.75%,　a　recall　of　75.41%,　and　an　F1　score　of　77.98%　for　the　hardest　test　data.　Conclusions:　In　this　paper,　we　proposed　a　distant　supervision　approach　to　develop　an　automatic　system　that　can　classify　high　and　low　suicide　risk　based　on　social　media　comments.　The　model　can　therefore　provide　volunteers　with　early　warnings　to　prevent　social　media　users　from　committing　suicide.