TL;DR - no.

Hypermedia is all the rage these days. New hypermedia APIs and standards are popping up all the time. But do you need to actually bother?

Hypermedia is hard. It's not just a case of adding URIs to your response and your done. There's all sorts of complexity if you are going to do it right, from the URL structure to the payload structure even to the declared media types.

If you are adding public API support onto an existing application that had an existing user base and an existing, or strong potential for, developer base then yes, hypermedia is probably a good idea.

If you're just starting, and have no product or users or anything yet, hypermedia is going to be more cost than benefit. It will slow you down, and make your end product more complicated to develop and maintain, and there will be no benefit. By the time you have all those things that make it a benefit, you'll understand your domain better and be better placed to do hypermedia properly.