In　this　article,　a　new　technique　for　macromodeling　of　high-frequency　circuits　and　components　called　high-order　deep　recurrent　neural　network　(HODRNN)　is　proposed.　This　technique　explores　an　alternative　approach　to　learn　RNN　for　time　dependencies　in　a　more　efficient　way　resulting　in　more　accurate　model.　HODRNN　uses　more　memory　units　to　track　previous　hidden　states,　all　of　which　are　returned　to　the　hidden　layers　as　feedback　through　various　weight　paths.　Moreover,　a　new　improved　structure　called　Hybrid-HODRNN　is　proposed　for　further　increasing　the　modeling　accuracy　of　HODRNN.　The　proposed　Hybrid-HODRNN　uses　hybrid　layers　with　both　single　and　high　orders　for　taking　advantage　of　HODRNN　and　also　reducing　the　overfitting　problem,　which　finally　leads　to　a　more　accurate　model.　In　addition,　the　proposed　method　requires　less　training　signals　compared　to　the　conventional　shallow　and　deep　RNNs　in　order　to　create　a　model　with　similar　accuracy.　Also,　the　obtained　models　from　the　proposed　method　are　considerably　faster　than　the　transistor-level　models　while　having　similar　accuracy.　By　modeling　three　high-frequency　circuits　in　this　article,　we　conclude　that　the　HODRNN　and　its　hybrid　structure　offer　the　ability　to　create　a　better　macromodel　of　high-frequency　nonlinear　circuits　than　the　conventional　RNNs,　which　verifies　the　superiority　of　the　new　macromodeling　techniques.