The question came before the Court in Bd. of Ed. v. Pico, and four Justices (led by Justice Brennan) took the view that "local school boards may not remove books from school library shelves simply because they dislike the ideas contained in those books." Four other Justices (led by Chief Justice Burger) expressly rejected this view (except in the narrow situation where the disagreement was based on pure partisanship, for instance if a Democrat-run board removed books because they were written by Republicans or because they praised Republicans). And the swing vote, Justice White, expressly refused to opine on this issue:
The plurality … issue[s] a dissertation on the extent to which the First Amendment limits the discretion of the school board to remove books from the school library. I see no necessity for doing so at this point….
[This case] poses difficult First Amendment issues in a largely uncharted field. We should not decide constitutional questions until it is necessary to do so, or at least until there is better reason to address them than are evident here.