--- aliases: [] --- #physic #compound-knowledge # [[Epistemic status]] #shower-thought #to-digest # Related > [!TODO] Related # TODO > [!TODO] TODO # Rulial space #todo https://arxiv.org/pdf/2101.10907.pdf - [[Stephen Wolfram]] #compound-knowledge-hole # What do you think [[GPT3]]? We are still at a very early stage in exploring the implications of this work for physics. We expect to make further progress in our future work, both in understanding the origins of the phenomena uncovered here and in relating them to models of fundamental physics. ## Introduction The goal of this paper is to help us better understand the fundamental structures of natural computation and information, as encoded in that class of machines known as Turing machines. Much of this work is inspired by a correspondence, initiated by the author, between a well‐known mathematician, Douglas Hofstadter, and the computer scientist, Peter Naur. The correspondence has been published in Scientific American (Hofstadter 1984). In his response to the article, Naur writes that “Hofstadter’s Law: ‘It always takes longer than you expect, even when you take into account Hofstadter’s Law.’” ### Analogy: Discrete Dynamical Systems Mathematicians have long attempted to develop formal models of “discrete dynamical systems” —that is, objects which evolve according to certain dynamics (such as those which govern particles through space and time), but whose evolution is governed by discrete state‐space rules rather than any continuous dynamics. As an example of such a model, one can consider the deterministic rules for the motion of a ball falling in gravitational fields, defined by Newton’s Laws. A more famous model is provided by the Turing machine. In both cases there is an abstract state space that is described by a fixed set of rules. One can then use these rules to investigate the dynamics of the system and its relation to other systems in the same class — e.g., to compare different systems and to approximate them using simpler ones. We adopt an analogous approach here for Turing machines. The principal goal in this paper will be to construct an abstract representation for each Turing machine, given some fixed set of rules describing its behavior; we will then explore ways in which one can manipulate this abstract model in a manner analogous to what one can do for other systems. To this end we will use our work on computing with words to developa new kind of computational abstraction that allows us to model both algorithms and implementations of algorithms entirely in terms of their abstract behavior on simple data structures (e.g., strings). This will provide us with a framework for analyzing how those abstract operations combine together to form complex algorithms, with interesting implications for both algorithmic complexity theory—for example, P versus NP—and for physical models of computation—for example, limits on speed‐ρ or decrease in required energy per bit as the number ρ million copies become infinitesimally small. ## Rule Systems as Computation ### The Classical Model: Finite Automata The first step towards understanding computation as a process occurring on a set of discrete objects is to choose an appropriate set of objects upon which one acts. One possible choice is that each object represents some kind of spatial configuration or position (e.g., the differences between configurations are all that matter). This leads us to consider discrete dynamical systems where each state represents such a configuration or position; such systems are called “finite automata” and are widely studied both in mathematics and computer science (Papadimitriou 1983; Erdös and Gallai 1966; Hopcroft et al., 1979; Knuth 1998; Dershowitz and Ziff 2002). A finite automaton consists of five pieces: a finite set S {0...n−1} called the state space, a set Sigma consisting either of characters or strings called input alphabet, another set called the tape alphabet consisting either of characters or strings representing “marks” on some “tape” device that can be written on and erased but cannot be read (or perhaps only those marks which represent characters), a transition function specifying how the current machine state changes according to input and current state markings on the tape, and finally an initial start state 0 together with a final accept/reject condition giving whether or not the current configuration satisfies some type of accept/reject condition (such as accepting all configurations represented by strings over Sigma ending in 0). For our purposes we will restrict ourselves to finite automata whose tape alphabet consists only of symbols from Sigma so that there are no additional marks that need to be handled explicitly outside Sigma (i.e., we will restrict ourselves to “deterministic Turing machines”). Infinite automata are also studied (e.g., Levitin 1967), but they are more difficult mathematically so we restrict our attention here to finite automata with nonempty tapes. Other variants include automata with no nondeterminism at all (as discussed below) and also automata with nondeterminism occurring only at special places within their transition functions (Kobler et al., 1997). Such special places are called “trap states” because they trap any attempt by one part of the computation from reaching another part before it has had time enough to reach it; these places allow Turing machines to implement nondeterministic processes inside their transition functions; see also Papadimitriou 1983 or Papadimitriou 1994 for discussion about trap states (for details see Dyer et al., 2011). The following illustrates an example: A = <Sigma, S> = {0,...,n-1} × {0,...m-1}m Transitions: <q0 ,q1>,<q0 ,q2>,<q0 ,q3>,<q1 ,q1>,<q1 ,q2>,<q1 ,q3>,<q2 ,q1>,<q2 ,q2>,<q2 ,q3>,<q3 ,q1>,<q3 ,q2>,<q3 ,q3> 0,a,b,c,0,0,0,0,0,0,0,0,1 0,b,c,a,1,1,1,1,1,1,1,1,0 0,c,a,b,2,2,2,2,2,2,2.3 ... The transition function maps pairs (Qi{0...n−1}, Qi+1) to triples (Qi+1{0...n−1}, ai∈Sigma + {E}* and Qi+2{0...n−1}). The empty string E* is used to indicate that no transition is possible. The initial start state is q0 and the accept condition is defined to be when the current state is qn and the tape has been completely read. This can also be written in terms of the transition function as a finite sequence of symbols from Sigma ending in the “accept” symbol 0. We restrict ourselves here to deterministic Turing machines for several reasons. First of all their transition functions are much simpler than for more general Turing machines; second of all by using only transitions from Sigma as transitions for our automata we eliminate any nondeterminism due to input symbols that do not occur on the tape; finally because there are no nondeterminism points we can easily construct finite automata that never halt (see Figure 1) or have loops (see Figure 2). These two simple examples can serve as models which give rise to many other Turing machines that are of interest. Figure 1: Infinite Neverending = lt;Sigma = {0} and Q = {q} and ∆ = {(q  q)} and q initial = 0 and F = {}>$ Figure 2: Infinite Looping = lt;Sigma = {x} and Q = {p} and ∆ = {(p  p)} and p initial = p and F = {}>$ When dealing with different types of Turing machines it may be convenient to use different sets for the various parts of the machine specification. We will often consider machines whose states are given by integers from 0... n−1; however sets like S or Q may be used instead. As usual we will use capital letters for set names and lower case letters for set elements. We will also use upper case characters for other math symbols such as > or ∈ (indicates an element of) or ⊆ (indicates a set that is contained in another set). If you prefer you can treat these as variables instead since any set may be substituted for them without altering the intended meaning. For example we could write: $T(qiqi+1)$ ∧ $T(qi+1qj) = T(qjqj+1)$ where qi and qj are assumed to be elements of Q. A Turing machine is said to halt on a given input string (for some accept condition) if it reaches a final state after reading the entire string. Note that this definition is quite different from the definition used in the literature; most authors use “halting” in a weaker sense which allows us to write nonfinal states as accepting states, thereby allowing Turing machines to be used as models of nondeterministic computation. ### The General Model: Pushdown Automata What formal properties would we like our automata to have? In general we would like our automata to be able to represent all possible algorithms, even those which contain nondeterministic choices or loops. We can do so, as was illustrated above by Turing’s machines, by using appropriate transition functions for our automata. However there are several other ways of constructing automata that will allow us to work with all possible algorithms. The most general class of automata we know of is pushdown automata. A pushdown automaton (PDA) is an abstract machine which has a stack L and a tape T as its “input”; the tape is written out with some initial configuration on it (see Figure 3). The stack and the tape are assumed to have certain properties; these are encoded in two new sets called the stack alphabet $SigmaL$ and the tape alphabet. Finally, as in Turing machines we need to specify a transition function $T(q, Λ, Γ, q', Γ')$ and an initial start state q0 together with an accept condition F. The transition function specifies how the current machine state changes according to input and current state markings on the tape and stack. The following illustrates an example: Figure 3: An Example of a PDA: One With Stack = lt;SigmaL = {a,b} and S = {0...m-1}>$ $T = <SigmaT ,S ,SigmaL ,S >$ Transitions: $(q0 ,a q1 ,A),(q0 ,b q1 ,B),(q0 ,c q0 ,E ),(q1 ,a q1 ,A ),$$(q1 ,b q2 ,B ),(q2 ,a q2 ,A ),(q2 ,b q2 ,B )$ $0,a,A,1$ $0,b,B,0$ ... The input tape may be viewed as a sequence of symbols from $SigmaT$ . The stack L is viewed as a sequence of symbols from $Sigmal$ . The symbol at the top of the stack is denoted by P (for “pushing”) and acts much like a push button on a mechanical calculator or on some software systems. Thus P = A represents pushing an A onto the stack; similarly P = B represents pushing a B onto the stack. In general we will assume that elements written on top of the stack cannot be erased or overwritten once they are written on top of the stack; thus when we write P = A on top of the tape we leave it there while we erase whatever was under it and write A below it. When $P = E$ (for “empty”), however, we erase whatever was under it and replace it with E . The following table summarizes some other properties of a PDA: Figure 4: Properties of $PDA = <SigmaT*,S*,SigmaL*,S*,T*,F*>$ Example where SigmaL* consists only of symbols from S* which are not in $SigmaT*$: $PDA = <{0},{},{0},{},<>,{}>$ Deterministic Turing Machine $PDA = <{a},{b},{a},{b},<a,b>0,1>$ Example where $SigmaL*$ consists of symbols which are both in and not in $SigmaT*$: $PDA = <{0},{},{0,c},{b},<>1,1>$ Deterministic Turing Machine $PDA = <{a},{b},{0,c},{b},<a,b>0,1>$ Note that these machines still have nondeterminism in the transition functions. In particular they can be Turing equivalent. The transitions here give the evolution of the stack $L$ and the tape $T$ according to the input sequence and current states $qi $and $P$ (for push). For example $(q0 ,a q1 ,A)$ means that if we are in state 0 with a written above the top of the stack and an A on top of the tape we will go to state 1 with A on top of the stack and the original A erased from the tape. Thus each transition represents a possible action that can be taken by our machine; in general we can choose what to write on top of the stack and what to erase from the tape. The transition function specifies how many times we can use each symbol in our alphabet over a given input; it could thus be viewed as a transition rate for each symbol $σ ∈ SigmaT*$ for each configuration $(qiqj ,P π ,Γ Γ ′)$. Note that no matter what we write on top of the stack we must eventually “pop” it by writing $P = E$ . If we were to write something else on top of it then the machine would terminate before having erased everything under it (see Figure 5). There is also another subtlety; we have assumed that all symbols written on the tape must be erased before another symbol can be written. This means that there is an upper bound on the number of symbols that may appear on top of the stack at any given time — if one pushes something onto a stack which has some symbols already there then they must all first be popped off before pushing something else. This last point is not really a restriction since this situation occurs seldom. Figure 5: Unsafe Pushdown Automata = lt;SigmaT*,S*,SigmaL*,S*,T*,F*>$ Similar topic links: [[The absurdity of Turing machine's local state]] [[Machine Learning Street Talk (MLST) - 82 - Dr. JOSCHA BACH - Digital Physics - DL and Consciousness UNPLUGGED]] [[Turing machine]] [[Low Kolmogorov complexity human reverse engineering tool]]