Deleting a folder in the Notes app is as simple as swiping left on the folder you want to delete. In the Notes app navigate to the Folders section and swipe right to left to open the Delete option. Tap the "Delete" button and the folder will be deleted. Be careful that you have selected the correct option because once you have deleted a folder it will be gone forever. Only deleted notes are saved in the Recently Deleted section.
Update: The swipe left option only works if you saved the folder directly to your iPhone. There doesn't seem to be a way to delete folders from iCloud.Update 2 (07/04/19):
In iOS 13 or later, tap the Edit button in the upper right hand corner, then tap the 3 dots in a circle next to the folder you want to delete. Tap Delete to remove the folder from your Notes app.