They're a cool fruit with tropical ancestry that evolved to colder climates. They attract almost no pests (only had a few japanese beetles on them this year) and are pretty disease free. The only insect that really goes for them is the zebra swallowtail and they're not up here since pawpaw is not native and they only eat the leaves of pawpaws.
You definitely want grafted trees to get superior fruit. You can grow seedlings, but they'll take longer to fruit and you need to be sure they're seeds from superior parentage. We have a short growing season so you definitely want the vigorous varieties with early ripening. You need 2 different varieties for pollination and they are only pollinated by flies and some beetles...no bees. Hand pollination is a better way to make sure you get a good harvest.
This guy is in Marlborough, MA and has a bunch. Here's a vid of some early varieties he recommends. I have a few of these. Nyomi's Delicious is one of my most vigorous growing trees so far.