Is there anything I can do about my sons father moving in and out of his ex girlfriends house with our son?

This is getting so old. My Sons father can't get over this girl and it's a poisonous relationship. When he first got joint custody my son was 9 months and he had lived with this girl for a year. When my son was about 1 1/2 she kicked them out. changing the locks and throwing everything on the front lawn that wasn't useful to her. She kept his washer and dryers his t.v.s couch bed..... He had to go rent a room. He rented that room for another year and moved back in with her. They were only there for 3 months. According to my sons father she told him that she was moving back in with her parents so he had to go but they were also broken up. He moved out again sleeping on his friends couch and occasionally his friend would give my son and him his bed. He is now renting another room. Our son is now 3 1/2 and him and his ex have been having dinner going to the beach with my son and according to my son spending the night at her place. His father assures me that he wouldn't be dumb enough to go back out with her even with all the following. I'm worried about this pattern. Do you think the courts can do anything? The man is 32 and I'm 23 with my own apartment much more stable then he is. I'm curious if I should try to get more custody? It's 50/50 right now or if I can get an order to keep them from moving back in with her. I don't think it's ok for my son to grow up around that type of relationship and all the last minute moving.