In　this　Letter,　we　propose　an　all-optical　diffractive　deep　neural　network　modeling　method　based　on　nonlinear　optical　materials.　First,　the　nonlinear　optical　properties　of　graphene　and　zinc　selenide　(ZnSe)　are　analyzed.　Then　the　optical　limiting　effect　function　corresponding　to　the　saturation　absorption　coefficient　of　the　nonlinear　optical　materials　is　fitted.　The　optical　limiting　effect　function　is　taken　as　the　nonlinear　activation　function　of　the　neural　network.　Finally,　the　all-optical　diffractive　neural　network　model　based　on　nonlinear　materials　is　established.　The　numerical　simulation　results　show　that　the　model　can　effectively　improve　the　nonlinear　representation　ability　of　the　all-optical　diffractive　neural　network.　It　provides　a　theoretical　support　for　the　further　realization　of　a　photonic　artificial　intelligence　chip　based　on　nonlinear　optical　materials.　(C)　2021　Optica　Publishing　Group