Click on a colored block to remove it from the current neighborhood, click on a clear block to add it.
Removed blocks will remain unlabeled until you edit a different neighborhood and attach them there.
To make a bigger addition, alt or command click to begin drawing a polygon, and alt or command click to end it.
By default, the selection will be added to the existing neighborhood, this can be changed with
r for redraw mode, and d for delete mode, and a to get back to add.
Another option is to turn on continuous mode with
c. Once this is enabled, all blocks moused over will be added (or deleted depending on the mode), hit c again to go back to polygon mode.
(delete neighborhood coming soon)