College enrollment among young Americans has been declining gradually over the past decade. And men are driving that decline—particularly at four-year institutions, according to a new report from the Pew Research Center.

About 1 million fewer young men are in college but only 0.2 million fewer young women. As a result, men make up 44 percent of young college students today, down from 47 percent in 2011, according to recent U.S. Census Bureau data.

This shift is driven entirely by the falling share of men who are students at four-year colleges.

At two-year colleges, which are largely community colleges, the drop in enrollment has been similar for men and women, so the gender balance has not changed much. Men represent 49 percent of students ages 18 to 24, up slightly from 48 percent in 2011.

Read more: Fewer young men are in college, especially at 4-year schools.