Caching refers to a temporary static file that is stored in order to speed up the retrieval of it’s data. This is helpful in cases where content is displayed as HTML but has scripts like PHP gathering information from the database.
Having cached versions of html files being delivered speeds up the time that it takes for the server to display a page.
The main reason to use a website caching tool is for speed. Because the dynamic elements are turned to static elements, web pages load much quicker.
WordPress uses a lot of PHP functions to call HTML code from different parts of the site. Depending on the template and other factors, this code can get pretty intensive. It is almost always better to use a website caching tool with WordPress so that the pages display quicker.
Just over a month ago, WP Engine, a WordPress specific hosting company that this website uses, said that because of their new infrastructure, Caching actually slowed down their website performance. Read their post about No Caching Plugins