Overview For this project, we will compare the price and trading volume of Bitcoins across different exchanges. The deadline for this project is Dec 2, 2020 (at 11:59pm). Bitcoins exchanges act as market makers between buyers and sellers. It is not uncommon for different exchanges to quote different prices at the same time. This does not necessarily lead to arbitrage opportunities because of bid-ask spreads, fees, and other frictions. In this project, we are interested in answering the following questions: 1. Do some exchanges consistently quote lower prices than others? 2. Which exchanges have the highest volume of bitcoin trading? These are some of the questions investors should consider when deciding where to buy/sell bitcoins. Note: For simplicity, we will only focus on prices and volume in this project. Of course, there are many other important aspects to consider when choosing where to trade bitcoins. These include trading fees, minimum trade size, and minimum initial investment. This project has 3 parts and is worth 100 points in total: 1. Part 1: Understanding the Cryptowatch API (20 points) a. Premiminaries b. Questions 2. Part 2: Creating a Cryptowatch API (80 points) a. Creating auxiliary functions b. Obtaining a list of exchanges c. Obtaining quotes 3. Part 3: Analysis (optional, 0 points) Important: This project is to be completed individually by each student. You should not collaborate. You are allowed to post questions about the project in the thread. I will create in the Discussion Board. However, please make sure you do not post partial solutions when asking questions, and you do not answer any questions posted by other students. I will answer any project-related questions myself. Given the number of students that forgot about this important rule, I will say it again. Please: 1. Do NOT post partial solutions or tentative code when asking questions in the discussion board. 2. Do NOT answer any questions posted by other students regarding this project. Many thanks in advance for your understanding. 1 Files You received a zipped file with the following contents: / | _project2.py | project2.pdf |___files/ | | | | |__ _testfiles/ | | test.json where represents your Zid and: • represents the main folder containing all the project files. For example, if your Zid is z1234567, this folder will be z1234567. • _project2.py contains the functions you need to write for this project. Some of the functions are already written, while others you will write yourself. Please see the instructions below for more information. • project2.pdf This file • files/: This is the directory where the files you download from Cryptowatch will be saved. You must include these files in your project submission General Instructions Important: Do not exchange complete or partial codes with other students. Preparing the files for this project 1. Copy the main folder to your computer. Make sure it is inside a directory in the Python system path (if you copy the folder into your PyCharm project folder, it will be). For example, if your zid is z1234567, this folder will contain the following files: z1234567/ | project2.pdf | z1234567_project2.py | |___files/ | | | | |__ _testfiles/ | | ... |... ... 2. Unless explicitly stated below, do not change any variable, import statement, function or parameter names in the _project2.py module. In particular, do not change the variables EXCH_LST, PERIOD, or SYMBOL, which will be described later. 3. Open the _project2.py module and change the variable PRJDIR, so it points to the location of the folder. In the example above, this variable should point to the location of the z1234567 folder. 4. Make sure the sub-folder called files is empty for now. During the project, you will download a series of files and save them inside this folder. Do not include other files inside this folder. All files inside this folder will be part of your submission and we will use them when assessing your project. 2 Instructions for Part 1 1. Read the Part 1 section below. Make sure you answer all the questions in the module _project2.py. Do not submit answers in any other format. We will parse your com￾pleted _project2.py file for the answers of both Parts 1 and 2. 2. I cannot emphasize this enough: Although the questions for Part 1 are included in this document, you should provide the answers in the _project2.py file! Instructions for Part 2 1. Follow the instructions to complete the remaining functions in the _project2.py module. 2. IMPORTANT After you are done, run the function main without any parameters (which will download the exchanges and OHLC files again). This will make sure the files folder contains the following files: • exchanges.json • One file for each exchange in EXCH_LST and the pair in SYMBOL These are the files that will be assessed. Other files in that folder will not be considered. Instructions for Part 3 Part 3 is optional and will not be marked (no bonus marks either). Project submission You should submit a zipped file containing the completed _project.py module and all the files in files/. The easiest way to do that is to compress the entire folder into a zip file once your are done with the project. This is the zip file you must submit Part 1: Understanding the Cryptowatch API During the first part of this project, you will familiarize yourself with Cryptowatch’s API and the data we will use in our analysis. Preliminaries: Cryptowatch is a trading platform that provides real-time market data for many cryptocurrency exchanges. The real-time data on Cryptowatch is provided directly from cryptocurrency exchanges. Cryptowatch allow users to access their data through a RESTful API. Free/anonymous access is limited, however. You can find more information about Cryptowatch’s API by reading the Documentation. OHLC Quotes In this project, we will collect open, high, low, and close prices for bitcoins trading in different exchanges. We often refer to these quotes collectively as OHLC, which stands for Open, High, Low, and Close. The OHLC quotes are always associated with a period of time so that: • Open: The quote at the beginning of the period • High: The highest quote during the period • Low: The lowest quote during the period • Close: The quote at the end of the period For instance, suppose we set the period to be “5 minutes”. The OHLC quotes for a five-minute period starting at midnight today would be obtained as follows: 3 1. Split the day into 5-minute intervals: first second third .... ----|---------|---------|-------|--------|--> Time 12:00am 12:05am 12:10am ..... now 2. Compute the open, high, low, and close prices for each interval 3. Return the data in some pre-specified data format, for instance: row timestamp open high low close 1 12:05 00:00:00 .. .. .. .. 2 12:10 00:00:00 .. .. .. .. .. .. .. .. .. .. The timestamp above refers to the end of the interval. This means that the first interval (the first data point) will start at 12:00am and end at 12:05am. The OHLC quotes for the first interval (row=1) would then represent1: • Open: The first quote immediately after 12:00am • High: The highest quote from 12:00am to 12:05am • Low: The lowest quote from 12:00am to 12:05am • Close: The last quote before (or at) 12:05am • First interval is (12:00am, 12:05am] • Second interval is (12:05am, 12:10am] This means that: - A quote at 12:00 00:00:01 would belong to the first interval - A quote at 12:05 00:00:00 would belong to the first interval - A quote at 12:05 00:00:01 would belong to the second interval In the context of OHLC quotes, each individual interval is sometimes referred to as a candle. This comes from the popular candlestick charts used to represent trading patterns. For example, the table above contains the “5-minute” candles for the previous 24-hours, where each row is a candle. Unix Timestamps A UNIX timestamp is an integer representing a point in time. The value of a UNIX timestamp represents the number of seconds that have elapsed since some arbitrary date and time (taken to be midnight of January 1, 1970, UTC), corrected for leap seconds. Cryptowatch uses these timestamps to represent the date/times that define OHLC intervals. Cryptowatch returns dates in UTC. You can use the datetime package to convert between these timestamps and local or UTC date/time. The relevant methods are fromtimestamp and utcfromtimestamp. If ts is an integer representing a Unix timestamp, these methods will return datetime instances so that: • .fromtimestamp(ts) --> datetime object (for local date/time) • .utcfromtimestamp(ts) --> datetime object (for UTC date/time) For instance, consider the unix timestamp 1474736400. What is the UTC and local datetime for this timestamp? import datetime as dt ts = 1474736400 1A valid question at this point would be: “In the example above, does the timestamp 12:05 00:00:00 belong to the first or second interval?” For this project, we will assume semi-open intervals (open at the beginning of the interval, closed at the end of the interval). For instance, in the example above: 4 # This is the local date for this timestamp local_dt = dt.datetime.fromtimestamp(ts) print(local_dt) # returns '2016-09-25 03:00:00' # This is the UTC date for this timestamp # (note that the method is "utcfromtimestap") utc_dt = dt.datetime.utcfromtimestamp(ts) print(utc_dt) # returns '2016-09-24 17:00:00' Questions (20 points) Please answer the questions below in the appropriate section of your _project2.py mod￾ule. This section is clearly defined in that module using the label “ANSWERS TO PART 1 QUESTIONS” For each topic below, read the relevant documentation and answer the following questions. 1. Rate limit: Assume all questions below refer to free/anonymous access to Cryptowatch’s API. • Q1: Do you need an API key for free/anonymous access? • Q2: What is the initial value of your allowance in Cryptowatch credits? • Q3: Do all requests cost the same in credits? 2. Assets • Q4: What an asset? – (one sentence, max 150 characters) • Q5: What is the symbol for the Bitcoin asset? 3. Pairs • Q6: What is a pair? – (one sentence, max 150 characters) • Q7: What is the symbol for the USD price of one Bitcoin (i.e., the Bitcoin, USD pair)? 4. Markets • Q8: What is a market? – (one sentence, max 150 characters) • Q9: What is the market endpoint URL for the pair “btcusd” and exchange “kraken”? Hint: make sure you use the pair btcusd! – (one url, max 150 characters) • Use any web browser to “navigate” to this endpoint. Given the raw data contained in the response, answer the following questions: – Q10: In which format is the raw data represented? – Q11: Based on the data contained in the response, is this market active? In other words, can you use USD to trade bitcoins in this exchange? – Q12: Based on the data contained in the response, what is the id for this market? This is the integer Cryptowatch uses to identify this market. 5. OHLC Candlesticks • Q13: What is the endpoint URL for the OHLC market represented by the Kraken exchange and the btcusd pair? Assume that you do not have any optional query parameters. • Q14: For the resources returned by OHLC endpoints (without query parameters), what is the length of the interval for the candles stored under the JSON key “14400”? Hint: The page OHLC 5 Candlesticks contains an example of a response to a request with an endpoint similar to the one above. You can use this response and the period table at the end of OHLC Candlesticks to answer this question. • Q15: Suppose you are given the following candle in a string: [ 1474736400, 8744, 8756.1, 8710, 8753.5, 91.58314308, 799449.488966417 ] Which of these values correspond to the close quote? • Q16: In the candle above, the unix timestamp is 1474736400. Use the datetime module to answer the following question: What is the UTC date and time corresponding to this timestamp? Part 2 During this part of the project, you will modify some functions in the _project2.py module you received. Important: As discussed above, Cryptowatch allows you to retrieve a certain amount of data for free for every 24-hour period. This amount is more than enough to complete this project inside a 24-hour period (it corresponds to hundreds of API requests). However, Make sure you do not make unnecessary requests and run out of credits! Making sure you do not run out of credit is part of the assessment for this project. You will not get an extension if you run out of credits. Setup All that is required is changing the value of the variable PRJDIR in the _project2.py so it points to the location where you extracted the Zid folder included in the zip file you received. This variable should end with your Zid. For instance, if your Zid is z1234567 the value of PRJDIR should be: # Linux/Mac PRJDIR = '/z1234567' # Windows PRJDIR = r'\z1234567' If your PRJDIR variable is not set correctly, the module will produce an error the first time you try to run it. Important Please do not modify any of the following: - Constants (the variables in all caps) with the exception of PRJDIR (and the questions for Part 1). - The name or parameters of any function. - Any function not listed in the outline below. Outline Note: The module you were given contains many test functions that you can use (they all start with _test. You can change these functions if you want. We have not given you the output of the _test functions for 6 obvious reasons. Below is the suggested outline you should follow to complete this part of the project: 1. Complete all the auxiliary functions • to_json: converts a Python object to a JSON string and save it to a file. Follow the instructions in the docstring. You can use the function _test_to_json to test this function. • from_json: Reads a file containing a JSON string and convert it to an object. Follow the instructions in the docstring. You can use the function _test_from_json if you want. • get_ohlc_loc: This function is already written and should not be modified. 2. Download and save a file containing information on all exchanges in Cryptowatch • write_exchange: Follow the instructions in the docstring. • read_exchanges: Read the contents of the file saved by write_exchanges and returns a dataframe with the result. • Note: The reason for this two-step approach is to minimize the number of API calls. You only have to run the function write_exchange once! After you have downloaded the list of exchanges, you should not call this function anymore. Instead, use read_exchanges. 3. Create a function to download OHLC quotes for a given market • write_ohlc: Follow the instructions in the docstring. • Call the _test_write_ohlc function and examine the contents of the file you downloaded. Once you are finishing debugging, do not call this function again (remember your credits are limited). • Open the file ...files/kraken_btceur.json and examine its contents. It should be similar to what is described in the write_ohlc docstring. 1. Create a function to produce a dictionary with the contents of a candle. • candle_to_dic: follow the instructions in the docstring. • You can use the function _test_candle_to_dic to test it. 1. Create a function to return the number of seconds in a given period label. • period_to_secs: follow the instructions in the docstring. Keep it simple. 1. Create a function to read the quotes for a given market and period. • read_ohlc: follow the instructions in the docstring. • You can use the _test_read_ohlc function to test it. 1. Create a function to read the quotes for a given market and period and return a dataframe. • mk_ohlc_df: follow the instructions in the docstring. • You can use the _test_mk_ohlc_df function to test it. 1. Create a function to return a list of exchange symbols • get_valid_exchanges: follow instructions in the docstring 2. Download the relevant data • get_all_data: follow the instructions in the docstring • Note: make sure you do not modify the value of the constants SYMBOL PERIOD and EXCH_LST. Part 3 This part will not be marked. It is optional but it will give you an idea of how this API can be used. 1. Uncomment and run the function compare_exchanges 2. Do you find that some exchange has consistently lower quotes? 3. Do some research and try to figure out why some exchanges have lower prices than others.