## Transcribed Text

Problem 1: Entropy
In this problem, be calculating what’s called Entropy. This is related, but not directly, to the
entropy you’ve seen in physics. Some time ago, Claude Shannon at IBM was working on making
a mathematical model of communication. Imagine, you’d have a piece of text, and then his
function would give a number indicating how important or significant it was.
What Shannon observed was that importance was really the degree of how surprising the
message was. So, in a sense, the message itself wasn’t as important as how unexpected it
was. He then realized that he could utilize probability as a measure of surprise:
Message Entropy Probability
not surprising ↓ high
surprising ↑ low
For this problem, we are only working with finite probabilities. We can think of probabilities
as a list of numbers p0, p1, . . . , pn such that
pi ≥ 0, i = 0, 1, . . . , n (1)
1 = p0 + p1 + p2 + · · · + pn (2)
=
Xn
i=0
pi (3)
Line (3) is usually how it’s written using the P as a shorthand for addition. We’ll learn about this
later, but I thought it might be interesting to see it now. Observe it looks a lot like range! You
might think about what’s similar and what’s different. To say in words, they are a finite collection
of numbers that are non-negative that sum to exactly one.
We can make a list (we’ll assume the items are of the same type and immutable) into a
probability. Consider x = ["a","b","a","c","c","a"]
1. gather the items uniquely (hint: dictionary)
2. count each time the item occurs
3. create a new list of the count/len(dictionary)
The list of probabilities would be y = [3/6,1/6,2/6].
We still need to show you how to calculate entropy:
entropy = −(p0 log2
(p0) + p1 log2
(p1) + · · · pn log2
(pn) (4)
entropy(y) = −(3/6 log2
(3/6) + 1/6 log(1/6) + 2/6 log2
(2/6) (5)
= −(.5(−1.0) + 0.17(−2.58) + .33(−1.58)) (6)
= 1.46 (7)
Because of continuity arguments we treat log2
(0) = 0. Python’s math module will correctly state
this math error, so you’ll have to simply add 0 if the probability is 0.
Listing 1: entropy.py
1 import math
2 def makeProbability(xlst):
3 #TO DO: IMPLEMENT FUNCTION
4
5 def entropy(xlst):
6 #TO DO: IMPLEMENT FUNCTION
7
8
9 s1 = [’a’,’b’,’a’,’b’,’a’,’b’,’b’,’b’]
10 s2 = [(1),(2),(3),(4)]
11 s3 = [1]
12 s4 = [1,2]
13 xlst = [s1,s2,s3,s4]
14 for i in xlst:
15 print(entropy(makeProbability(i)))
Output function.py
0.954434002924965
2.0
0.0
1.0
Deliverables Problem 1
• Complete the two TO DO’s.
• Put this code in a new module named entropy.py.
Problem 2: Magick
You’ve encountered a soothsayer named Soothy McSoothface. He will magically determine any
whole, positive number you guess by asking you to perform a few operations and tell him the
result. It’s quite amazing. Modify the previous code to show he’s not so magic after all.
Magical Encantation
Pick a number x
Add fifteen to x
Multiply the sum by three
Subtract nine from the product
Divide the difference by three
Subtract 12 from the quotient
Hocus Pocus–that is your number
Table 1: Encantation (operations) on a secret number that yields the secret number at the end.
To give you a start, here’s the code and where you’ll write yours:
1 # Input value: some number
2 # Output value: the result of the magical encantation
3 def magic(x):
4 #TODO: implement the magical encantation
5
6 if __name__=="__main__":
7 #get input
8 x = input("Pick any positive whole number: ")
9
10 #change from string to integer
11 x = int(x)
12
13 print("Your number was", magic(x))
Ponder this a bit, and try your best. When you’re done, and it works, take a look on the next
page. To help you out, we’ll remind you of some of the vocabulary:
Operation Result Name
Division Quotient
Addition Sum
Multiplication Product
Subtraction Difference
Power Exponetiation
Table 2: Vocabulary for mathematical operations.
Deliverables Problem 2
• Complete the program.
• Put your program in a new module named magic.py

These solutions may offer step-by-step problem-solving explanations or good writing examples that include modern styles of formatting and construction
of bibliographies out of text citations and references. Students may use these solutions for personal skill-building and practice.
Unethical use is strictly forbidden.

import math

def makeProbability(xlst):

# TO DO: IMPLEMENT FUNCTION

# gather the items uniquely (hint: dictionary)

unique = set(xlst)

# count each time the item occurs

counter = [0.0] * len(unique)

# iterate over the main list

for item in xlst:

# get index and value the unique list

for index, val in enumerate(xlst):

# if this value equals to the current item in the list

# count it up

if val == item:

counter[index] = counter[index] + 1.0

break

pass

pass

pass

# create a new list of the count/len(dictionary)

for index, val in enumerate(counter):

# get probability of each unique item

counter[index] = (counter[index]) / float(len(xlst))

pass

return counter

pass...