A　torsional　pendulum　device　containing　hyperbolic　tangent　input　nonlinearities　can　be　formulated　as　a　nonaffine　system.　Unlike　basic　affine　systems,　the　optimal　feedback　control　of　complex　nonaffine　plants　is　difficult　but　quite　important.　In　this　paper,　the　approximate　optimal　control　design　of　continuous-time　nonaffine　nonlinear　systems　is　investigated　with　the　help　of　reinforcement　learning.　For　addressing　the　learning　algorithm　conveniently,　an　effective　pre-compensation　technique　is　adopted　to　perform　proper　system　transformation.　Then,　the　integral　policy　iteration　strategy　is　incorporated　to　relieve　the　demand　of　system　dynamics.　Moreover,　the　actor-critic　structure　is　implemented　by　virtue　of　neural　network　approximators.　Finally,　the　experimental　verification　for　the　proposed　torsional　pendulum　plant　is　conducted　after　a　learning　process　of　20　iterations　and　the　stability　performance　with　basic　robustness　guarantee　can　be　observed　during　two　case　studies.　(C)　2019　Elsevier　Ltd.　All　rights　reserved.