Deleting a folder in the Notes app is as simple as swiping left on the folder you want to delete. In the Notes app navigate to the Folders section and swipe right to left to open the Delete option. Tap the "Delete" button and the folder will be deleted. Be careful that you have selected the correct option because once you have deleted a folder it will be gone forever. Only deleted notes are saved in the Recently Deleted section.
Update: The swipe left option only works if you saved the folder directly to your iPhone. There doesn't seem to be a way to delete folders from iCloud.