Learn back-end development by writing real code

Boot.dev Blog ยป Computer-Science ยป Examples of Heuristics in Computer Science

Examples of Heuristics in Computer Science

By Winston Wagner on Nov 30, 2020

Heuristics in computer science and artificial intelligence are “rules of thumb” used in algorithms to assist in finding approximate solutions to complex problems. Often, there’s simply too much data to sift through to come to a solution promptly, so a heuristic algorithm is used to trade exactness for speed. However, because heuristics are based on individual rules unique to the problem they are solving, the specifics of the heuristics vary from problem to problem.

Heuristics aim to produce solutions in a reasonable time frame that are good enough for solving the problem at hand. The solution produced using a heuristic may not be the perfect or exact solution, but it’s valuable as an approximate or best-guess solution. Some problems would require hundreds of thousands of years for an exact answer, but we can produce an approximate solution almost instantly.

Heuristic Trade-Offs

The entire value proposition of a heuristic is based on trade-offs. Typically we are trading accuracy for time. That said, there are several different levers we have to pull when designing a good heuristic.

  • Optimality: Many problems have multiple solutions, for example, “what is a good path to get from city A to city B? Do we need the best path, or will a good path be good enough?
  • Completeness: When there are multiple valid solutions to a problem do we need to find all of them? Will a subset of valid solutions suffice?
  • Accuracy: Many questions don’t have a correct answer. For example, “Will Tommy like a pair of boots or a pair of gloves for Christmas?” A heuristic can improve accuracy in these situations.
  • Execution time: The primary goal of a heuristic is to provide a quick answer that’s good enough. Some heuristics are only marginally quicker than classic methods.

Example problems and some of their common heuristics are given below.

Traveling Salesperson Problem (TSP)

The TSP is a famous algorithm with a Big-O complexity of O(n!) and asks the question:

Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city and returns to the origin city?

For a low number of cities, this question could be reasonably brute-forced. However, as the number of cities increases, it becomes increasingly difficult to come to a solution.

The nearest-neighbor (NN) heuristic solves this problem nicely: the computer always picks the nearest unvisited city next on the path. NN does not always provide the best solution, but it is close enough to the best solution that the difference is often negligible for the purpose of answering the TSP. By using this heuristic, the Big-O complexity of TSP can be reduced from O(n!) to O(n^2).

Get a back-end job without spending $10k on a bootcamp

  • Learn Python, Javascript and Go
  • Build the professional projects you need to land your first job
  • Spend about 6 months (when done part-time)
  • Pricing as low as $24/month*
  • No risk. Cancel anytime.

Knapsack Problem

The knapsack problem poses the issue:

Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible.

An example heuristic for this problem is a greedy algorithm, which sorts the items in descending order of value per weight, and then proceeds to insert them into the “sack”. This ensures the most valuably “dense” items make it into the sack first.

Search Optimization

Search engine optimization has been sought after for as long as search engines have been around. Individuals using search engines want to find the information they are looking for as swiftly as possible. With such an incredible amount of information available, search engines must utilize heuristics to expedite the search process. At the start, a heuristic could try each possibility at each step, but as the search continues, it can choose to stop the search at any time if the current possibility is worse than the best solution already located. In this way, the search engine can be optimized for speed and correctness.

A simple path to your career in back-end development

The pace of Boot.dev's JavaScript, Python and Go courses has been perfect for me. The diverse community in Discord is a blast, and other members are quick to help out with detailed answers and explanations.

- Daniel Gerep from Cassia, Brasil

Applying Heuristics to Your Algorithms

To apply heuristics to your algorithms, you need to know the solution or goal you’re looking for ahead of time. If you know your end goal, you can specify rules that can help you achieve it. If the algorithm is being designed to find out how many moves a knight can make on a square, 8x8 chessboard while visiting every square, it’s possible to create a heuristic that causes the knight to always choose the path with the most available moves afterward.

However, because we’re trying to create a specific path, it may be better to create a heuristic that causes the knight to choose the path with the fewest available moves afterward. Since the available decisions are much narrower, so too are the available solutions, and so they are found more quickly.

Learn back-end without spending $10,000

  • Write modern code in JavaScript, Python and Go
  • Build and deploy real backend projects to your personal portfolio
  • Compete in the job market by mastering computer science fundamentals

Find a problem with this article?

Report an issue on GitHub