you an expert on cryptocurrency and ladies and gentlemen without further ado please help me give a warm Malaysian welcome to undress you you you [Music] you [Music] [Music] welcome to the first lecture in our series on Bitcoin and crypto currencies I want to start by introducing the four lecturers who are going to be speaking in the series the first lecturer is Joseph Bono he’s a postdoctoral researcher in computer science at Princeton University the second lecturer is me Edie Felton I’m a professor at Princeton in computer science and in the Woodrow Wilson School the third lecture is Arvind Narayan and he’s a computer science professor at Princeton and fourth our special guest lecturers Andrew Miller he’s a PhD student in computer science at the University of Maryland worked for the cryptocurrency stuff so if you came for the crypto currency stuff let me assure you first of all that we will get to it in this lecture and that having laid the groundwork in this lecture there’s going to be a lot more specifically crypto currency focused material in later lectures all right so let’s get to it [Music] in segment 1.1 we’re going to talk about cryptographic hash functions we’ll talk about what they are and what their properties are and then later we’ll move on and talk about what their applications are so a cryptographic hash function is a mathematical function and it has three attributes that we need to start with first of all a hash function can take any string as input absolutely any string of any size it produces a fixed size output will use 256 bits in this series of lectures because that’s what Bitcoin does and it has to be efficiently computable meaning given a string it in a reasonable length of time you can figure out what the output is [Music] [Music] welcome to the first lecture in our series on bit peeking in the series the first lecturer is Joseph bono he’s a postdoctoral researcher in computer science at Princeton University the second lecture is me Edie Felton I’m a professor at Princeton in computer science and in the woodwose researcher in computer science at Princeton third lecture is Arvind Narayan a second computer science at Princeton a professor at pre and fourth our special guest lecturers Andrew Miller he’s a PhD student in computers there will be 11 lectures in this lecture number one we’re going to do two things first we’ll introduce some cryptographic primitives that turn out it will be necessary for talking about crypto currencies in particular this talk about cryptographic hashes and digital signature talk about some of the ways in which and then at the end of the lecture we’ll start talking about crypto currencies and I’ll give some examples illustrate some of the design challenges I want to apologize for covering the cryptographic material and beginning unfortunately we have some of our vegetables a little bit in order to much pull a groundwork for the crypto guys so if you came for the critical currency stop you let me show you first of all that we will get to it in this lecture a little bit and that having laid the groundwork in this lecture there’s going to be a lot more if you specifically crypto currency focused material in later lectures that we will get to it right so let’s get to it [Music] one point one we’re going to talk about cryptographic cryptographic hash function that is a mathematical function and it has three attributes that we need to start later first of all a hash function can take any string as input grabs nearly any string of any mathematical produces a fixed size output we’ll use 200 series of lectures and it has to be efficiently computable meaning given a reasonable we’re going to need hash functions either cryptographic using the cryptographic properties of hash functions are a complicated topic in general we’re going to focus here on three particular projects and I’ll explain in a minute what those particular function is collision-free so that it has a hiding property each of these I’ll talk about what the property is what it means why it’s useful to have a first collision so the first property that we want from a function and what that means is that it’s impossible nobody can find values x and y so the first such that x and y are different and yet the hash of x is equal to the hash right what that means is that and so if we look at the operations on the function as depicted by these arrows here’s X and H of X here’s Y and H of Y that nobody can find a situation like this you have an x and y that are separate and yet when you have the same H of X now one thing to notice is that I said nobody has to be collisions doing this and understand why that is we can use this diagram over here on the left I’m depicting all the function which can be anything and over here I have all the possible outputs which has to be so the right-hand side here the outputs there are only over here there are more possibilities and so if you think that every point over here on the left is going to be them out by an arrow some point on the right you can see that as you go from all the points over here on the left into the right multiple values over here on the Left isn’t mounted the same in fact in general there will be a very large number of possible inputs mapped to any particular point so collisions doing said before nobody can find a collision and that’s the key a question we know collisions exist the question is are there any collisions that are my regular people using right now to make things even worse I said that it has to be impossible that winda collisions we tell you how to find a collision because there’s a method that’s guaranteed to work and the method works like this to the 130 randomly chosen inputs over on the Left clouds that previous diagram that at least two of them are going and so this is a very simple method one finding a collision it works no matter but of course the problem is that this takes it very very long extreme to do the 130 and that’s of course an astronomic works.we matter which hash function we’re using there’s still a ninety-nine point eight percent probability that this works and if it doesn’t work this doesn’t really matter and the reason it doesn’t really matter is that this procedure takes two to the 130 steps in order to get to that high probability so we can say something like this we can say that if every computer every maze ever made by humanity was computing since the in order to beginning of the entire universe up to something like the odds that if they would have found a collision it’s still infinitesimally small what’s so serious that it’s way less than the to be destroyed by a giant meteor engine is still infinity so we know how to find a collision but this method takes two one the question is is there some other methods that could be used on a particular hash table in order to find a combination and that’s the question that is harder to is there a faster way to find collisions well for some possible values of hash functions of course there are for example if our hash function authorities simply take the input modulo 256 then we would know an easy way to find input would be the value 3 + 3 + 2 – 2 4 6 so for some possible values of the hash function it’s very easy to value 3 for others we don’t know read + 2 to the 240 now one thing I need this to note is that there’s no hash constant which has been proven to be positionality they’re just the people have tried really really hard to find so we choose to believe which is bad those are collision-free they’re just the people have tried really really hard what good does the collision freedom do on so if we can assume that we have a hash function there is collision operation then we can use that hash function as a message digest and what I mean by that is good following that if we know then it’s safe to assume the next message because if someone knew and I’ve seen wives are different they had the same there’s not a collision as we know you knowing the hashes are the same of supersu and this lets us use the hashes kind of message digest suppose for example that we had filed a really big fuss and we wanted to be able to recognize later whether another file is the same and the file we saw the first suppose for example that we had so one way to [Music] [Music] welcome to the first lecture in our series on Victorian Krypton I want to start by introducing those poor lecturers who are going to be welcome to the first line first lecture is Joseph no he’s a postdoctoral researcher the second lecturer is me Edie Felton I’m a professor at Princeton in computer science and into Woodrow Wilson School the third lecture is Arvind Narayan and he’s a computer science professor at Princeton and fourth our special guest lecturers Andrew Miller he’s a PhD student in computer science at the University of Maryland there will be 11 lectures in total in this lecture number one we’re going to do two things first we’ll introduce some cryptographic primitives that turn out to be necessary for talking about crypto currencies in particular we’ll talk about cryptographic hashes and digital signatures and we’ll talk about some of the ways in which those are used to build crypto currencies and then at the end of the lecture we’ll start talking about crypto currencies and I’ll give some examples of simple crypto currencies that illustrate some of the design challenges that we need to deal with I want to apologize for covering the cryptographic material at the beginning unfortunately we have to eat some of our vegetables a little bit in order to let to lay groundwork for the cryptocurrency stuff so if you came for the cryptocurrency stuff let me assure you first of all that we will get to it in this lecture and that having laid the groundwork in this lecture there’s going to be a lot more specifically cryptocurrency focused material in later lectures all right so let’s get to it [Music] in segment 1.1 we’re going to talk about cryptographic hash functions we’ll talk about what they are and what their properties are and then later we’ll move on and talk about what their applications are so a cryptographic hash function is a mathematical function and it has three attributes that we need to start with first of all a hash function can take any string as input absolutely any string of any size it produces a fixed size output will use 256 bits in this series of lectures because that’s what Bitcoin does and it has to be efficiently computable meaning given a string it in a reasonable length of time you can figure out what the output is so that’s a hash function but we’re going to need hash functions that are cryptographically secure the the cryptographic properties of hash functions are a complicated topic in general but we’re going to focus here on three particular properties and I’ll explain in a minute what those are in particular that the function is collision free that it has a hiding property and that it’s puzzle friendly and for each of these I’ll talk about what the property is what it means and then I’ll talk about why it’s useful to have a function that has that property so first collision free so the first property that we need from a cryptographic hash function is that it’s collision free and what that means is that it’s impossible nobody can find values x and y such that x and y are different and yet the hash of x is equal to the hash of y and so if we look at the operation of the function as depicted by one of these red arrows here’s X and H of X and here’s Y and H of Y that nobody can find a situation like this that you have an x and y that are separate and yet when you hash them they’d hash to the same value now one thing to notice is that I said nobody can find I didn’t say that there is no collision because if you think about it there has to be a collision collisions do exist and to understand why that is we can we can use this diagram over here on the Left I’m depicting all of the possible inputs to this function which is any size for their bride so here I have saved the possible outputs which has to be a string of 256 bits in size so the right-hand side here the outputs there are only 2 to the 256 possibilities over here there are more possibilities and if you think that every point over here on the left is going to be mapped by an arrow to some point on the right you can see that as you go from all the points over here on the left into the right it has to get crowded and in fact that there have to be multiple values over here on the left that mapped to the same output over here in fact in general there will be a very large number of possible inputs that map to any particular output so collisions do exist I said before nobody can find a collision and that’s the key question we know collisions exist the question is are there any collisions that are findable by regular people using regular computers okay now to make things even worse I said that it has to be impossible to find a collision let me tell you how to find a collision because there’s a method that’s guaranteed to work and the method works like this that we’re going to pick two to the 130 randomly chosen inputs over on the left cloud of that previous diagram and if we pick those two to the 130 randomly chosen inputs it turns out there’s a ninety-nine point eight percent chance that at least two of them are going to collide and so this is a very simple method for finding a collision it works no matter what the hash function is but of course the problem is that this takes a very very long time to do you have to compute the hash function to to the 130 times and and that’s of course an astronomical number this method works no matter which hash function we’re using there’s still a ninety-nine point eight percent probability that this works and if it doesn’t work just try it again it’ll probably work the next time but this doesn’t really matter and the reason it doesn’t really matter is that this procedure takes two to the 130 steps in order to get to that high probability so we can say something like this we can say that if every computer every mate ever made by humanity was computing since the beginning of the entire universe up to now the odds that they would have found a collision is still infinitesimally small so small that it’s way less than the odds that the earth will be destroyed by a giant meteor in the next two seconds which didn’t happen okay so we know how to find a collision but this method takes too long to matter the question is is there some other method that could be used on a particular hash function in order to find a collision and that’s the question that is harder to to answer is there a faster way to find collisions well for some possible values of hash functions of course there are for example if our hash function were to simply take the input modulo 2 to the 256 that is it just selected the last 256 bits of the input then we would know an easy way to find a collision one collision would be the values 3 + 3 + 2 to the 256 so for some possible values of the hash function it’s very easy to find a collision for others we don’t know now one thing I need to note is that there’s no hash function in existence which has been proven to be collision free there are just some that people have tried really really hard to find collisions and haven’t succeeded and so we choose to believe that those are collision free ok now what good does collision freedom do us if we can assume that we have a hash function that is collision free then we can use that hash function as a message digest and what I mean by that is the following that if we know that x and y have the same hash then it’s safe to assume that x and y are different because if someone knew an x and y that were different that had the same hash of course that would be a collision since there’s not a collision that we know of then knowing the hashes are the same we can assume that the values are the same and this lets us use the hash as a kind of message digest suppose for example that we had a file a really big file and we able to recognize later whether another file was the same as the file we saw the first time right so one way to do that would be to save the whole big file and then when we saw another file later just compare them but because we have hashes that we believe are collision free it’s more efficient to just remember the hash of the original file then if someone shows us a new file and claims that it’s the same we can compute the hash of that new file and compare the hashes if the hashes are the same then we conclude that the files must have been the same and that gives us a very efficient way to remember things we’ve seen before and recognize them again and of course this is useful because the hash is small it’s only 2 256 bits while the original file might be really big Pasha’s you Elizabeth died and we’ll see later on in this lecture and in subsequent lectures why it’s useful to use hashes and message digests so the second property that we want from our hash function is that it’s hiding and the property that we want is something like this that if we’re given the output of the hash function that there’s no feasible way to figure out what the input X was the problem is that this property doesn’t exactly hold and to understand why that’s the case let’s look at this example so here what we’re going to do is an experiment where we flip a coin and if the result of the coin flip was heads we’re going to return the hash of the string heads and if the result was tails we’re going to return the hash of the string tails and now we’re going to ask someone who didn’t see the coin flip but only saw this hash output to figure out what the string was that was hashed that of course is going to be easy it’s easy in this scenario to find what the input string was it’s easy to find X you simply compute the hash of the string heads and the hash of the string tails and you see which one you got and so in just a couple of steps you can figure out what X was so the reason this example failed that is the reason why an adversary was able to guess what the string was was that there were only a couple of possible values of X and so if we’re going to have a property like this it needs to be the case that there’s no value of X which is particularly likely that is all the X has to be chosen from a set that’s in some sense very spread out so this method for the adversary of just trying all the possible values of X or just trying a few values of X that are especially likely is not going to work so the hiding property that we are going to need to set up is a little bit more complicated and the way we’re going to fix this problem with the common value X like heads and tails is we’re going to take the X and we’re going to put next to it we’re going to concatenate with it with it a value R which is chosen from a distribution that’s really spread out and so this H of R concatenated with X that means take all the bits of R and put after them all the bits of X and so what we’re going to say is given the hash of our together with X that it’s infeasible to find X and that this will be true in the formally stated property that if R is a random value chosen from a distribution that has high min entropy then given the hash of our concatenated with X it’s infeasible to find X and what is time e an entropy mean well it captures this intuitive idea that R is chosen from a distribution that’s really spread out and what that means specifically is that there’s no particular value that r could have had that would occur with more than a negligible probability so for example if R is chosen uniformly from among all of the strings that are 256 bits long then any particular string was chosen with probability 1 in 2 to the 256 which is truly a negligible value so as long as R was chosen that way then the hash of our concatenated with X is going to hide X and that’s the hiding property that the hash function will be deemed to have okay now let’s look at an application of that hiding property and in particular what we want to do is something called a commitment and this is kind of the digital analogy of taking a value a number and sealing it in an envelope and putting that envelope out on the table where everyone can see it now when you do that you’ve committed to what’s in the envelope but you haven’t opened it it’s secret from everyone else later you can open the envelope and get out the value but it’s sealed so commit to a value and reveal it later we want to do that in a digital sense so to be more specific about what is the API that we’re going to provide here the commitment API looks like this that there are two things you can do first you can commit to a message and that’s going to return two values a commitment and a key think of the commitment as the envelope that you’re going to put on the table and the key as a secret key for unlocking the envelope then later you allow someone else to verify given a commitment and given a key which you’ve told them in the meantime and the message they can verify that that commitment key and message really do go together and this will return true or false okay now to seal MSG in an envelope what we do is we commit to the message and that returns a commitment and a key and then we publish the commitment that’s putting the envelope on the table now later to open the envelope what we’re going to do is publish the key and the message that we committed to and then anybody can use this verify call with the commitment that we published previously the key and message that we just announced to check the validity of of our opening the envelope okay and the property of course we want from this is that it behaves like sealing an envelope and in particularly the two security properties are these first give and calm the commitment the envelope on the table that someone just looking at the envelope can’t figure out what the message is the second property is that it’s binding that when you commit to what’s in the envelope you can’t change your mind later that is it’s infeasible to find two different messages such that you can commit to one message and then later claim that you committed to another and the whole thing will verify okay so how do we know that these two properties hold well first we need to talk about how we’re going to actually implement commitments and the way we’re going to implement commitments is like this that in order to commit to a value message we’re going to generate a random 256 value bit value and call it the key and then we’re going to as the commitment returned the hash of the key concatenated together with the message and as and as the key value we’re going to return h of this of the visca T and then later to verify someone is going to compute this same hash of the key they were given concatenated with the message and they’re going to check whether that’s equal to the commitment that they saw okay so this is a way of using hash functions both in the commitment and in the verification so now the security properties if we go down to the security properties that were at the bottom of the previous slide and we just plug in the definitions of how we’re going to implement this here that is this used to say comm given comm infeasible defined message we just plug in what comm is calm is that hash of key concatenated with message and similarly down here this is what happens when we take what was written there before and plug in the definition of verify in calm okay so now what these properties become the first one is given H of key concatenated with message it’s infeasible to find message well it turns out that that’s exactly the hiding property that we talked about before ki was chosen as a to random 256-bit value and therefore the the hiding property says that if we take the message and we put before it something that was chosen from a very spread out distribution like I said a random 256-bit value then it’s infeasible to find the message so this is exactly the hiding property and this one down here turns out to be exactly the the collision-free property so that if someone can find two messages which have the same hash like this well then they have an input value here and an input value there that are different and yet those have the same hash and so because of the two security properties we’ve talked about for hashes so far this commitment scheme will work in the sense that it will have the necessary security properties okay so that’s the second security property of hashes that they’re hiding and the application of that is commitments the third security property we’re going to need is that their puzzle friendly and this is again a little bit more complicated but let me just go through it a bit by bit that for any possible output value Y that you might want from the hash function we’re going to use Y is an output value of the hash later that if K is chosen from a distribution that has high mean entropy that is K is chosen randomly from some set that’s super spread out then there’s no way to find an X such that the hash of K and X is equal to Y so what this means is basically that if someone wants to target the hash function if they want it to come out to some particular output value Y that if there’s part of the input that is chosen in a suitably randomized way that it’s very difficult to find another value that hits exactly that target so the application we’re going to use of this is we’re going to build a search puzzle and what that means is we’re going to build a mathematical problem which requires searching a very large space in order to find the solution and where there’s no shortcuts a way to find a good solution other than searching that large space that’s that’s a search puzzle to be more specific the idea is that if we’re given a puzzle id which is chosen from some high min entropy distribution that is some very spread out probability distribution and we given a target set why which someone tries to wants to make the hash function fall into then we want to try to find a solution X so that if we hash the puzzle ID together with this solution X we get a result that’s in the set Y so the idea is y is a target range or set of hash results that we want ID specifies a particular puzzle and X is a solution to the puzzle and the puzzle friendly property here implies that there’s no solving strategy for this puzzle which is much better than just trying random values of X and so if we want to if we want to pose a puzzle that’s difficult to solve that we can do it this way as long as we can generate puzzle IDs in a suitably random way and we’re going to use that later when we talk about Bitcoin mining that’s a sort of computational puzzle we’re going to use ok so we’ve talked about three properties of hash functions and one application of each of those now let me talk just very briefly about the particular hash function we’re going to use there are lots of hash functions in existence but this is the one Bitcoin uses and it’s a pretty good one to use it’s called sha-256 or sha-256 and it works like this basically it takes the message that you’re hashing and it breaks it up into blocks that are 512 bits in size the message isn’t going to be in general necessarily exactly a multiple of the block size so we’re going to add some padding at the end and the padding is going to consist of at the end of the padding a 64-bit length field which is the length of the message in bits and then before that it’s going to consist of a 1 bit followed by some number of 0 bits and you choose the number of 0 bits so that this comes out exactly 2 to the end of a block so once you’ve padded the message so that its length is exactly a multiple of the of the 512 bit block size you then chop it up into blocks and you then execute this computation you start with the 256 bit value called the IV that’s just a number that you look up in a standards document and then you take the IV and the first block of the message you take those 768 total bits and you run them through this special function C and the compression function and out comes 256 bits you now take that with the next 512 bits of the message run it through C again and you keep going each iteration of Si crunches in another 512-bit block of the message and mixes it in sort of logically to the to the to the result and when you get to the very end you have consumed all of the blocks of the message plus the padding the result is the hash that’s a 256 bit value and it’s easy to show that if this function see this compression function is collision-free then this entire hash function will also be collision free the other properties are a little bit more complicated so I won’t talk about them here okay so we’ve talked about hash functions we’ve talked about what hash functions do we’ve talked about three properties of hash functions and applications of those properties and the specific hash function that we use in Bitcoin in the next lecture segment we’ll talk about ways of using hash functions to build more complicated data structures that are used in distributed systems like Bitcoin in section 1.2 we’re going to talk about hash pointers and their application a hash pointer is a kind of data structure that turns out to be used a lot in the systems that we’re talking about and a hash pointer is basically a simple thing that we’re going to take a pointer to where some information is stored and we’re going to together with that pointer store a cryptographic hash of the information so whereas a regular pointer gives you a way to retrieve the information a hash pointer is going to let us ask to get the information back and it’s also going to let us verify that the information hasn’t changed so hash pointer tells us where something is and what its value was okay and we’re going to draw a hash pointer in diagrams like this then we’re going to have we’re gonna have h of and then an arrow that points to something so anything drawn like this they think of it as being a hash pointer to this this thing it’s a pointer to where it’s stored and it’s also the hash of the value that this data had when we last saw it and we can take hash pointers and we can use them to build all kinds of data structures so a key idea here take any data structure a linked list a binary search tree or something like that and implement it with hash pointers instead of pointers as we normally would for example here’s a linked list that we’ve built with hash pointer and this is a data structure that we’re going to call a blockchain so just like a regular linked list where you have a series of blocks and each block has data as well as a pointer to the previous block in the list here the previous block pointer will be replaced with a hash pointer so it says where it is and what the value of this entire previous block was and we’re going to store we’re going to remember the head of the list like this just as a regular hash pointer okay and a use case for this for a blockchain like this is a tamper evident log that is if we want to build a log data structure that stores a bunch of data so that we can add data on to the end of the log but if somebody goes later and messes with data that is earlier in the log we’re going to detect it that’s what the tamper evidence means so to understand why a blockchain gives us this tamper evident property let’s ask what happens if an adversary wants to go back and tamper with data later that’s in the middle of the chain okay so let’s assume that an adversary wants to tamper with this block down here he wants to change the data here and he wants to do it in such a way that we the holders of the of the hash pointer at the head here won’t be able to detect it all right so the adversary changed the contents of this block and therefore the hash here which is a hash of this entire block is not going to match up because the hash function is collision free it must be the case that the hash of this block is now different and so we could detect the inconsistency between this data and the hash pointer that we remembered before or we could do that in unless the adversary also tampers with the hash pointer if he tampers with this hash pointer then he makes these to match up but now he’s changed the content of this block and what that means is that when we come back later and hash the contents of this block it’s not going to match the hash that we remembered before because the contents of the block has changed and so we’re going to detect the inconsistency between this the contents of this block and this hash unless the adversary also tampers with the block over here on the right but now when he does that the hash of this block is not going to match the hash that we remembered up here and the hash that we’re holding on to and this the adversary can’t tamper with because this is the value that we remembered as being the head of the list and so the upshot of this is that if the adversary wants to tamper with data anywhere in this entire chain in order to keep the story consistent he’s going to have to tamper with the hash pointers all the way back to the beginning and he’s ultimately going to run into a roadblock because he won’t be able to tamper with the head of the list and so what this means is that just by remembering this hash pointer we’ve essentially remembered a kind of hash a tamper-evident hash of the entire list all the way back to the beginning and so we can build a block chain like this containing as many blocks as we want going back to some special block at the beginning of the list which we might call the Genesis block and that’s a tamper evident log built out of a block chain now another useful data structure that we can build using hash pointers is a binary tree we can build a binary tree with hash pointers and this is called in the jargon a Merkle tree after ralph merkle who invented it and the idea is this suppose we have a bunch of data blocks which will draw across the bottom down here we’re going to take pears consecutive pairs of these data blocks and for these two data blocks we’re going to build a data structure here that has two hash pointers one to each of these blocks and similarly all the way across will then go another level up and this this block here will will contain a hash pointer of these two children down here and so on all the way back up to the root of the tree and then just like before we’re going to remember just the hash pointer up here at the head of the tree and we can then if we want traverse down through the hash pointers to any point in Milton we can make sure that data hasn’t been tampered with just like I you