Turns out my neighbor is a bit of a bee expert. Who knew? Anyway, he's says they're relatively harmless at the moment. They've settled about 15 feet up in a pine, all huddled around the queen. He says they'll be there of a day or two while the scouts go out and look for a spot to establish a new hive. He said they'll probably swam again around dawn.
Last year I had bold-face hornets build a hive on a bird feeder. They say they are pretty aggressive, but the feeder was out of the way, so I let them be. It was pretty cool watching the hive grow in size all summer long. It was probably about the size of two footballs by September.