---
aliases: [entropy]
---
#physic #mathematic #ai
# [[Epistemic status]]
#shower-thought #to-digest
# Related
- [[Second law of thermodynamics]]
- [[The asymmetry of second law of thermodynamics]]
- [[Quantum level three multiverse|quantum multiverse]]
- [[Kolmogorov complexity]]
- [[Physic/Complexity]]
- [[Physic/Thermodynamics]]
- [[Readwise/Articles/towardsdatascience.com - The Intuition Behind Shannon’s Entropy]]
- [[Physic/Entropy, energy, assembly]]
# TODO
> [!TODO] TODO
# Entropy
- [[Second law of thermodynamics]]
- [[Big bang]]: from disorder is born order
<iframe width="560" height="315" src="https://www.youtube.com/embed/eX9NpXH7UrM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
- [[Physic/Time|time]] from no-time
- [[Space]] from no-space
- Matter from no-matter
- Life from no life
- The earth digest energy into higher entropy
[[Schrodinger]] highlights as early as 1945 the paradox of the living organization, which does not seem to obey the second principle of thermodynamics.
![[Screenshot 2022-07-20 at 12.47.06.png]]
>What life depends upon is the fact that the Sun is much hotter than the darkness of space, and consequently the photons from the Sun have a considerably higher frequency (namely that of yellow light) than the infra-red photons that Earth returns to space
>~ [[Roger Penrose]]
Sun turns low entropy energy into higher entropy energy which is turned into even higher entropy energy by organic systems on earth
![[Pasted image 20220818075356.png]]
![[Screenshot 2022-07-20 at 12.50.21.png]]
## Shannon Entropy
$H(X) = - \sum_{i=1}^n p_i log(p_i)$
```py
import math
import random
def H(sentence):
entropy = 0
# There are 256 possible ASCII characters
for character_i in range(256):
Px = sentence.count(chr(character_i))/len(sentence)
if Px > 0:
entropy += - Px * math.log(Px, 2)
return entropy
# The telegrapher creates the "encoded message" with length 10000.
# When he uses only 32 chars
simple_message ="".join([chr(random.randint(0,32)) for i in range(10000)])
# When he uses all 255 chars
complex_message ="".join([chr(random.randint(0,255)) for i in range(10000)])**
In [20]: H(simple_message)
Out[20]: 5.0426649536728
In [21]: H(complex_message)
Out[21]: 7.980385887737537
# The entropy increases as the uncertainty of which character will be sent increases.
```