A　formation　matrix,　which　is　presented　to　identify　the　positions　of　the　mobile　robots　and　state　space　with　lower　dimension,　is　constructed.　By　analyzing　the　merit　and　demerit　of　various　formations,　state　criterion　and　reinforcement　function　are　provided.　The　effect　on　the　learning　result　owing　to　uncertain　partition　of　sensed　area　is　weakened.　Simulation　experiment　shows　that　the　presented　approach　is　feasible.　Finally,　some　deficiencies　are　analyzed.