Once you take on a hunt in Dauntless you are thrust into a match to tackle a specific enemy. Depending on whether you use a standard hunt or a solo hunt in Dauntless, you’ll start an instance that can’t really be quit out of. There are a couple of options to abandon a hunt though.
The first thing you can do to abandon a hunt is to do so before you’ve actually departed to an island for the hunt. This can be done by pressing the right thumbstick while finding a server. This will allow you to abandon a hunt you’ve queued up for prior to it starting. The screen you’re looking for is the one in the image above. This is the screen that will appear after pressing the thumbstick, but it will only stay up for a short bit.
Once you’ve moved to the airship staging phase there isn’t a way to abandon the hunt. You’ll need to wait to you land at the island to quit out by using the return to main menu option. Other than this, there is no other way to abandon a hunt in Dauntless.