Funding is one of the major differences. Finances for public colleges mainly come from the state but can also come from the federal government. Finances for private colleges comes from private governing boads and tuition.
Private colleges usually cost more to attend and are smaller in size. They also treat students the same with regards to tuition expenses and admission - regardless to whether they are residents of the state or international students. Public colleges usually offer less of a competition for admisstion to state residents - some offer guaranteed admission to high school graduates with a particular GPA. However, they are more stringent with students from out of state.
Some private colleges also are affiliated with a particular religion or organization while a public college is not.
Also, some consider private colleges to offer a higher quality of education because they have the finances available to hire the best professors.
From personal experience I can point out a few other differences I noticed. I went to a private college that did not offer dorms nor were there fraternities and sororities. There was not a highly developed sports program and a lot of energy was focused on academia. That's not to say that all private colleges are like that. In fact all the above are now available at the Macon, GA branch of the college I attended here in Atlanta.
I also went to a public high school but when it was evident that the teachers knew less than some of the students

my mother pulled me out and put me in a private high school. It was affiliated with the Baptist so one thing that was missing from my high school experience was the prom. The Baptist did not condone dancing so I never attended a spring dance or prom equivalent until I went to college. Interestingly enough, the college I went to was also Baptist affiliated. We had dances there but no alcohol was allowed.