It＇s　a　promising　way　to　use　Unmanned　Aerial　Vehicles　(UAVs)　as　mobile　base　stations　to　collect　data　from　sensor　nodes,　especially　for　large-scale　wireless　sensor　networks.　There　are　a　lot　of　works　that　focus　on　improving　the　freshness　of　the　collected　data　or　the　data　collection　efficiency　by　scheduling　UAVs.　Given　that　sensing　data　in　certain　applications　is　time-sensitive,　with　its　value　diminishing　as　time　progresses　based　on　Timeliness　of　Information　(ToI),　this　paper　delves　into　the　UAV　Trajectory　optimization　problem　for　Maximizing　the　ToI-based　data　utility　(TMT).　We　give　the　formal　definition　of　the　problem　and　prove　its　NP-Hardness.　To　solve　the　TMT　problem,　we　propose　a　deep　reinforcement　learning-based　algorithm　that　combines　the　Action　Rejection　Mechanism　and　the　Deep　Q-Network　with　Priority　Experience　Replay　(ARM-PER-DQN).　Where　the　action　rejection　mechanism　could　reduce　the　action　space　and　PER　helps　improve　the　utilization　of　experiences　with　high　value,　thus　increasing　the　training　efficiency.　To　avoid　the　unbalanced　data　collection　problem,　we　also　investigate　a　variant　problem　of　TMT　(named　V-TMT),　i.e.,　each　sensor　node　can　be　visited　by　the　UAV　at　most　once.　We　prove　that　the　V-TMT　problem　is　also　NP-Hard,　and　propose　a　2-approximation　algorithm　as　the　baseline　of　the　ARM-PER-DQN　algorithm.　We　conduct　extensive　simulations　for　the　two　problems　to　validate　the　performance　of　our　designs,　and　the　results　show　that　our　ARM-PER-DQN　algorithm　outperforms　other　baselines,　especially　in　the　V-TMT　problem,　the　ARM-PER-DQN　algorithm　always　outperforms　the　proposed　2-approximation　algorithm,　which　suggests　the　effectiveness　of　our　algorithm.