As their name implied, the Puritans were interested in stripping away all of the things that the centuries had added to the faith of Christianity. Ceremonies and public celebrations didn't really interest them. They knew what was important, and the most important thing to know about Jesus is that he rose from the dead. And his resurrection is the most important message of hope that Christianity provides.
Easter is the day that we remember that Jesus returned. And if it takes parades, hiding easter eggs, eating chocolate bunnies to help remember, that's fine. But the Puritans remembered it every day. They didn't need Easter to remind them.