Occasions　such　as　stalled　vehicles　or　crashes　led　by　abnormal　trajectories　should　be　instantly　identified　and　then　dealt　with　quickly　by　the　city　traffic　management　system　for　the　sake　of　road　safety.　However,　a　fast　and　accurate　automatic　detection　system　based　on　machine　learning　in　general　meets　with　great　challenges　from　the　shortage　of　recorded　accident　data,　resulting　in　low　detection　accuracy.　Many　existing　studies　implement　a　two-level　detection　approach:　stalled　vehicles　are　detected　at　the　stationary　level,　while　abnormal　trajectories　are　detected　at　the　mobile　level.　This　paper　proposes　a　novel　triple-layer　framework　to　distribute　these　two　levels　to　three　parallel　layers　for　maximum　efficiency.　A　straightforward　background　extraction　algorithm　is　applied　at　the　beginning　of　this　framework　for　motion-stationary　distribution.　Layer　1　implements　a　lightweight　optical-flow-based　feature　extraction　algorithm　to　convert　the　mobile　visual　features　to　learnable　data.　With　a　clustering　algorithm　that　learns　the　common　trajectories　in　an　unsupervised　manner,　abnormal　trajectories　are　detected　in　Layer　2.　Simultaneously,　in　Layer　3,　a　custom-trained　object　detection　algorithm　is　applied　to　detect　the　stall/crashed　vehicles.　The　computational　efficiency　is　improved　and　the　detection　accuracy　is　boosted.　Experiments　conducted　on　Nvidia　AI　City　Challenge　Dataset　demonstrate　the　effectiveness　of　our　LRATD　(Lightweight　Real-Time　Abnormal　Trajectory　Detection　framework)　in　terms　of　104%　gain　in　detection　speed　compared　to　the　fastest　entry,　while　achieving　0.935　S4-Score,　only　2.1%　less　than　the　current　state-of-the-art　method.　Overall,　the　performance　of　LRATD　opens　the　possibility　of　its　real-life　application.