While　physics-informed　neural　network　(PINN)　has　shown　promise　in　solving　partial　differential　equations　(PDEs),　their　performance　is　limited　by　the　expressive　capacity　of　standard　neural　network　architectures,　particularly　when　dealing　with　complex,　nonlinear　problems.　To　improve　the　nonlinear　expressive　capacity　of　the　network　and　broaden　the　applications　of　PINN,　we　propose　a　double-activation　neural　network　(DANN)　for　solving　parabolic　equations　with　time　delay,　where　each　neuron　is　equipped　with　two　activation　functions　and　a　new　parameter　is　introduced　in　one　of　the　functions　to　form　quadratic　terms.　To　address　the　issue　of　low　fitting　accuracy　caused　by　the　discontinuity　of　solution＇s　derivative,　a　piecewise　fitting　approach　is　proposed　by　dividing　the　global　solving　domain　into　several　subdomains　according　to　the　discontinuous　points.　The　convergence　of　the　loss　function　is　proved.　We　conduct　a　series　of　numerical　experiments　to　show　the　efficiency　of　the　proposed　DANN　technique,　including　solving　delay　partial　differential　equations　(DPDEs)　with　constant　delay,　time-dependent　delay,　and　delay　differential　equations　(DDEs)　with　state-dependent　delay.　The　robustness　of　DANN　is　assessed　by　varying　the　number　of　training　points,　hidden　layers,　neurons　per　layer,　and　random　seeds.　We　also　evaluate　the　extended　application　of　DANN　by　simulating　a　second-order　rogue　wave　in　the　derivative　nonlinear　Schrödinger　(DNLS)　equation,　to　show　its　applicability　beyond　DPDEs.　©　2025　Elsevier　B.V.