You need bait in order to catch bigger fish. Bait is attached to the end of your fishing line when you throw back a small or medium-sized fish. Throwing back a large or huge fish will not give you bait; the fish will simply swim away. There are three bait types. The type of bait received depends on the size of the fish that you throw back.
Level 1 bait is attached to the rod by default and can be used to catch small fish.
Level 2 bait is attached when a small fish is thrown back, and it can be used to catch small or medium-sized fish. Wearing the Bait Hat makes this your default bait.
Level 3 bait is attached when a medium-sized fish is thrown back, and it can be used to catch a fish of any size (small, medium, large or huge).