Teachers　tend　to　set　the　free-text　questions　for　testing　the　comprehensive　ability　of　students.　That　leads　to　the　increasing　attention　to　the　intelligent　auto-grading　system　for　easing　the　grading　load　on　examiners.　In　this　paper,　we　present　a　novel　automatic　essay　scoring　system　based　on　Natural　Language　Processing　and　Deep　Learning　technologies.　In　particular,　the　proposed　system　encodes　an　essay　as　sequential　embeddings　and　harnesses　a　bi-directional　LSTM　to　catch　the　semantic　information.　Meanwhile,　the　system　constructs　the　attention　for　each　essay　so　that　the　network　can　learn　to　focus　on　the　valid　information　correctly　in　an　article,　which　can　also　provide　the　reasonable　evidence　of　the　predictive　result.　The　dataset　for　training　and　testing　is　the　public　essay　set　available　in　the　Automated　Student　Assessment　Prize　on　Kaggle.　The　study　shows　that　our　system　achieves　state-of-the-art　performance　in　grade　prediction,　and　more　importantly,　our　intelligent　auto-grading　system　can　focus　on　the　critical　words　and　sentences,　analyze　the　logical　semantic　relationship　of　the　context　and　predict　the　interpretable　grades.