Computable and non-computable problems in TOC

Computable Problems – You are familiar with many problems (or functions) that are computable (or decidable), meaning there exists some algorithm that computes an answer (or output) to any instance of the problem (or for any input to the function) in a finite number of simple steps. A simple example is the integer increment operation:

f(x) = x + 1 

It should be intuitive that given any integer x, we can compute x + 1 in a finite number of steps. Since x is finite, it may be represented by a finite string of digits. Using the addition method (or algorithm) we all learned in school, we can clearly compute another string of digits representing the integer equivalent to x + 1. Yet there are also problems and functions that are non-computable (or undecidable or uncomputable), meaning that there exists no algorithm that can compute an answer or output for all inputs in a finite number of simple steps. (Undecidable simply means non-computable in the context of a decision problem, whose answer (or output) is either “true” or “false”). 

Non-Computable Problems – A non-computable is a problem for which there is no algorithm that can be used to solve it. The most famous example of a non-computability (or undecidability) is the Halting Problem. Given a description of a Turing machine and its initial input, determine whether the program, when executed on this input, ever halts (completes). The alternative is that it runs forever without halting. The halting problem is about seeing if a machine will ever come to a halt when a certain input is given to it or if it will finish running. This input itself can be something that keeps calling itself forever, which means that it will cause the program to run forever. Another example of an uncomputable problem is: determining whether a computer program loops forever on some input. You can replace “computer program” with “Turing machine or algorithm” if you know about the Turing machine. 

Proving Computability or Non-Computability – We can show that a problem is computable by describing a procedure and proving that the procedure always terminates and always produces the correct answer. It is enough to provide a convincing argument that such a procedure exists. Finding the actual procedure is not necessary (but often helps to make the argument more convincing). To show that a problem is not computable, we need to show that no algorithm exists that solves the problem. Since there are an infinite number of possible procedures, we cannot just list all possible procedures and show why each one does not solve the problem. Instead, we need to construct an argument showing that if there were such an algorithm, it would lead to a contradiction. The core of our argument is based on knowing the Halting Problem is non-computable. If a solution to some new problem P could be used to solve the Halting Problem, then we know that P is also non-computable. That is, no algorithm exists that can solve P since if such an algorithm exists, it could be used to also solve the Halting Problem, which we already know is impossible. The proof technique where we show that a solution for some problem P can be used to solve a different problem Q is known as a reduction.A problem Q is reducible to a problem P if a solution to P could be used to solve Q. This means that problem Q is no harder than problem P since a solution to problem Q leads to a solution to problem P. 

Some Examples Of Computable Problems – These are four simple examples of the computable problem:

1. Computing the greatest common divisor of a pair of integers.
2. Computing the least common multiple of a pair of integers.
3. Finding the shortest path between a pair of nodes in a finite graph.
4. Determining whether a propositional formula is a tautology.

Some Examples Of Computable Problems – State Entry Problem. Consider the problem of determining if a string ‘w’ is given to some Turing machine ‘M’ will it enter some state ‘q'(where q belongs to a set of all states in Turing machine M and string w is not equal to an empty string). Is it computable or non-computable? 

Explanation – We show the State Entry Problem is non-computable by showing that it is as hard as  The Halting Problem, which we already know is non-computable. State Entry Problem is asking us on given string w if we start from initial state of Turing machine will it reach to a state q. Now this state entry problem can be converted to a halting problem. Halting problem is whether our Turing machine ever halts, and the state entry problem is asking the same thing whether this Turing machine halts at some state q if we give string w as input to the Turing machine M. So the state entry problem is non-computable as we converted it to the halting problem which we already know is non-computable problem. So in this way, we can prove non-computability.

Computable Problems:

Computable problems are those that can be solved by an algorithm, a step-by-step procedure that can be executed by a computer. These problems have well-defined inputs and outputs, and there exists a deterministic algorithm that can produce the correct output for any given input.

Advantages:

 Efficiency: Computable problems can be solved efficiently using algorithms. There exist efficient algorithms for many computable problems, allowing for practical solutions.
Predictability: Since computable problems have well-defined inputs and outputs, their behavior is predictable. This makes it easier to reason about their properties and analyze their performance.

Disadvantages:

Limitations: Not all problems are computable. There are certain problems, such as the Halting Problem, which cannot be solved by any algorithm. These problems have undecidable properties, meaning there is no algorithm that can always produce the correct output.
Complexity: Although computable problems can be solved, the efficiency of the algorithms can vary significantly. Some problems may have solutions that are computationally expensive, requiring a large amount of time or resources to compute the result.

Non-computable Problems:

Non-computable problems are those that cannot be solved by any algorithm. These problems often involve undecidable properties or require infinite computational resources.

Advantages:

Flexibility: Non-computable problems encompass a wide range of interesting and challenging questions that cannot be answered by algorithms. They can represent fundamental questions in mathematics and logic, stimulating research and exploration.
Theoretical Insights: Non-computable problems often lead to deep theoretical insights and help establish the boundaries of what can be computed. They push the limits of our understanding of computation and inspire the development of new computational models.

Disadvantages:

Lack of Practical Solutions: Since non-computable problems cannot be solved by algorithms, there is no general method to obtain their solutions. This limits their practical applicability in real-world scenarios.
 Intractability: Non-computable problems can be inherently complex and often involve infinite computations. This makes it difficult to reason about their properties or derive meaningful results.