The first federal official of any kind to be impeached and removed from office was a federal judge from New Hampshire named John Pickering. He was an appointee of President George Washington and was generally aligned politically with the Federalists. After Thomas Jefferson was elected president in 1800, the Jeffersonians went on the attack against what they saw as untoward Federalist influence over the federal courts.
Pickering was vulnerable. He had faced credible past accusations of both mental instability and drunkenness. Did they count as "Treason, Bribery, or other high Crimes and Misdemeanors," which is what Article II, Section 4 of the Constitution requires for a judge to be "removed from Office"? Or perhaps he had run afoul of Article III, Section 1, which states that "the Judges, both of the supreme and inferior Courts, shall hold their Offices during good Behavior." Either way, a majority of Congress wanted him out and the requisite two-thirds majority was present in the Senate to do it. Pickering got the boot in 1803.
That set the stage for the real showdown over the power to impeach judges. Emboldened by the successful removal of Pickering, the Jeffersonians turned their glare on Supreme Court Justice Samuel Chase. An outspoken Federalist, Chase especially drew the ire of the Jeffersonians because of his role as the presiding trial judge in several Sedition Act prosecutions carried out by the Federalist administration of President John Adams.