First, realize that no matter where you chose you're going to hear glowing reviews from one person, and the next will tell you how awful it is. It's that way with most restaurants at WDW
Any 1 credit restaurant at WDW is okay for kids of any age...it's WDW! So if you want to go to Chefs, go to Chefs. My best advice is that if you're uncomfortable, make it an early reservation or a lunch reservation. People who are older or couples will tend to make later reservations. For example, a family might eat at 5:30, my boyfriend and I will try for 7. So your family will be leaving by the time my boyfriend and I are sitting down. Then you won't worry about your 2.5 year old. Now, it's that way everywhere else in the world, but really, at Disney, if you're eating at 7 with a 2.5 year old, it's no big deal (I'd still go early, but that's JMO.)
I really enjoyed Chefs. I'd go ahead and book it, especially since your other meals are 2 character meals...and well, 50s might as well be They're very over the top and it will probably be nice just to go to a restaurant who's only "thing" is good food!
...and btw: loved Chefs, think Le Cellier is overrated. Now, someone else might post and tell you something completely opposite. So just go with what you think is right for your family!