Microstructure　significantly　affects　materials＇　physical　properties.　Predicting　and　characterizing　temporal　microstructural　evolution　is　valuable　and　helpful　for　understanding　the　processing-structure-property　relationship　but　is　rarely　conducted　on　experimental　data　for　its　scarcity,　unevenness,　and　uncontrollability.　As　such,　a　self-designed　in-situ　tensile　system　in　conjunction　with　a　scanning　electron　microscope　was　adopted　to　observe　the　grain　evolution　during　the　tensile　process.　We　then　used　a　deep　learning-based　model　to　capture　grain　growth　behavior　from　the　experimental　data　and　characterize　grain　boundary　and　orientation　evolution.　We　validated　the　framework＇s　effectiveness　by　comparing　the　predictions　and　ground　truths　from　quantitative　and　qualitative　perspectives,　using　data　from　(1)　a　tensile　experimental　dataset　and　(2)　a　phase-field　simulation　dataset.　Based　on　the　two　datasets,　the　model＇s　predicted　results　showed　good　agreement　with　ground　truths　in　the　short　term,　and　local　differences　emerged　in　the　long　term.　This　pipeline　opened　an　opportunity　for　the　characterization　of　microstructure　evolution　and　could　be　easily　extended　to　other　scenarios,　such　as　dendrite　growth　and　martensite　transformation.