Cryptocurrency-predicting RNN intro – Deep Learning w/ Python, TensorFlow and Keras p.8

Welcome to part 8 of the Deep Learning with Python, Keras, and Tensorflow series. In this tutorial, we’re going to work on using a recurrent neural network to predict against a time-series dataset, which is going to be cryptocurrency prices.

Text tutorials and sample code: https://pythonprogramming.net/cryptocurrency-recurrent-neural-network-deep-learning-python-tensorflow-keras/

Discord: https://discord.gg/sentdex
Support the content: https://pythonprogramming.net/support-donate/
Twitter: https://twitter.com/sentdex
Facebook: https://www.facebook.com/pythonprogramming.net/
Twitch: https://www.twitch.tv/sentdex
G+: https://plus.google.com/+sentdex

117 comments

  1. Harsath Zuckonit on

    I am from India planning to study in USA for undergrad so preparation for SAT is going on I have a question from you, only if I get into top universities I could get best resources to learn machine learning and deep learning or all stuffs are available online even I get into intermediate universities Please sentex replay

    Reply
  2. Douglas Urbano on

    Hey, i saw a rly good way to make a btc bot with drqn or rainbow from openai, maybe you can cover that some day, if u want i have some code about it!! Keep up with ur good work!!

    Reply
  3. Sfundo Mhlungu on

    Greetings from South Africa, I watch your vids even though I am not into python, I am c guy, I wish you can cover c#/ ++ one day, quality vids as always

    Reply
  4. Franz Tinuviel on

    Hi sentdex, have you been into a situation that you wanted to start a series about something but you found out that its too hard and way above of what you can do?

    Reply
  5. Mike G on

    This is the first time I’ve ever seen someone use pythons map method instead of pandas apply method to map/apply a function to a dataframe.

    Reply
  6. Victor Maricato on

    One of the first programming videos I understood the thinking behind it while you were writing code. (Opposite of, eg. Siraj videos)

    Reply
  7. Fish Sandwich Studio on

    Love these bro, keep up the good work! Only thing I noticed is you probably need to remove the last 3 records as their prediction will be wrong / out of context

    Reply
  8. Daniel Santos on

    Will you apply LSTM to this example? That would be great or you can finish this example and then apply this exactly example to a LSTM network, and compare the performance of both approaches. keep going with your great work.

    Reply
  9. UnlikelyMustard on

    The model you’re going to end up with is going to be a static model meaning that it wont keep re-training every minute with the correct price (ie: incremental training). Is there anyway to do this or are you maybe planning on discussing this?

    Reply
  10. Daniel Browning on

    BOOM!!!
    df = pd.merge(df, df2, on=’time’, how=’left’).fillna(method=’ffill’)
    df = pd.merge(df, df3, on=’time’, how=’left’).fillna(method=’ffill’)
    df = pd.merge(df, df4, on=’time’, how=’left’).fillna(method=’ffill’)

    Reply
  11. Hamza Abbad on

    Since I started working on my project about cryptocurrencies prediction everyone started to do the same! well, at least, their ideas are very basic, but yours seems interesting, so will you use the history of multiple coins to predict one of them?

    Reply
  12. Yusuf ÖZER on

    Could you try reinforcement learning for trading?RNN’s and other Machine Learning Algorithms are more efficient for long term prediction.But i think RL is the best for short therm prediction.Thanks for this video 🙂

    Reply
  13. Steve Bennett on

    Thanks for this tutorial. I have beeen looking for something like this for a while. I typically use R with multivariate models. Have been trying to transfer to python. This is a massive help.

    How can I make u a donation

    Reply
  14. Jason Wolbrom on

    Why can’t you take the 4 files separately train the rnn to have 1 output weight which can predict whichever coin you want. Lets say you have hundreds of coins and factors the dataframe itself could be to terabytes of training data.

    Reply
  15. Rivaan Bechan on

    I’ve been building something like this for a while now. Thank you for this, it is definetly refining my approach 🙏🏽 I’ve been struggling to incorporate fundamental/sentiment analysis into it, do you have any thoughts on how to incorporate that into the dataset? Can’t wait to see where this takes us 🙏🏽

    Reply
  16. Jeffrey Wallace on

    Just out of curiosity, do you dive right into coding or do you build any outlines, flowcharts, or diagrams ahead of time? It seems you use your written tutorials for your videos but is there any ground work prior to starting the written tutorial?

    Reply
  17. Dr. Computers on

    Hello, thank you for sharing your knowledge of PyQt4, I have already watched your videos and found it interesting to pass a project that I have already up and running in WxPython to pass to PyQt, as I am having the trouble to make a button1 (“select all “) let trace 20 checkbox in just one click .checked == True, and conversely, make button2 (” uncheck All “) uncheck those 20 checkboxes leaving = .checked == False, could you help me in this ? sending source code of + or – 240 lines …

    Reply
  18. Mikhil Mistry on

    @sentdex! I’ve been trying to reach you for the Cap!? I know you’re busy so just wanted to ask if I can get one? I will support you thru BTC/PayPal, etc.

    Reply
  19. FactCheckPolice on

    @sentdex! I’ve been trying to reach you for the Cap!? I know you’re busy so just wanted to ask if I can get one? I will support you thru BTC/PayPal, etc.

    Reply
  20. M on

    @sentdex! I’ve been trying to reach you for the Cap!? I know you’re busy so just wanted to ask if I can get one? I will support you thru BTC/PayPal, etc.

    Reply
  21. νικος παπανικολαου on

    i am in the process of learn data science.seeing how comfortable you are typing commands and how good you are what are you proposing is the best way to learn python and data science?I am taking notes and writing everything down so that i can remember it in the future but i feel like it takes me a long time. do you thing it would be better just to work on projects?

    Reply
  22. Johannes Wenisch on

    main_df = pd.concat([pd.read_csv(fn, index_col=0, usecols=[0, 4, 5], names=[col + “_” + re.search(‘/([^/.]*).csv’, fn).group(1) for col in [“TIME”, “CLOSE”, “VOLUME”]]) for fn in glob.glob(‘./crypto_data/*.csv’)], axis=1)

    I do not recommend this but it is possible. ^^ This is for macOS though. Requires glob+pandas+re.

    Reply
  23. Fusewithmusic on

    If you want to display more columns, use the following:

    with pd.option_context(‘display.max_rows’, 12, ‘display.max_columns’, 5):
    print(main_df)

    Reply
  24. mcan on

    Hey Harrison, you have merged all four dataframes but ETH -USD time column isn’t parallel with other three ones. Also row counts are all different.

    Reply
  25. Dave Unfairbank on

    the only way to win at finance is to either not be in it, or do binary options against tesla when elon musk tokes a joint on joe rogan. or use other peoples money…

    Reply
  26. Oscar The Jack Russell on

    maybe you should do more on alerts than trying to create trading bots, a simple alert that is fired off when the probablity of a preditcable pattern occuring say (70-85%) is a much better way to sart of trading

    Reply
  27. TypicalHog on

    Why are we merging all the data? I just can’t get my head around that.
    Is it because we want the network to learn the interaction between them?

    Reply
  28. Chidanand murthy on

    After merging the dataset, I am getting the error “columns overlap but no suffix specified: Index([‘BTC-USD_close’, ‘BTC-USD_volume’], dtype=’object’)”, am I the only one facing this issue? what am I doing wrong here?

    Reply
  29. Suleiman Mustafa on

    I am getting SyntaxError: invalid syntax when I use f string: dataset = f”crypto_data/{ratio}.csv” and my Python version is 3.6.5

    ^

    Reply
  30. Harsath Zuckonit on

    If you are the fan of OS Library Check this Out:-

    import pandas as pd
    import os

    SEQ_LEN = 60 #This is for 60 Mins
    FUTURE_PRED_PERIOD = 3 #Mins
    RATIO_TO_PRED = “LTC-USD”

    DIR_DATA = “crypto_data/”

    def classification_for_stock(current_price,future_price):
    if float(current_price) < float(future_price): return 1 else: return 0 #Importing Out Data(Saved by WEB SCRAPING) files_name = ['BTC-USD','LTC-USD','ETH-USA','BCH-USD'] files_path = ['BTC-USD.csv','LTC-USD.csv','ETH-USD.csv','BCH-USD.csv'] df_main = pd.DataFrame() for nums,file in enumerate(files_path): all_datasets = os.path.join(DIR_DATA,file) data1 = pd.read_csv(all_datasets,names=['time','low','high','open','close','volume']) data1.rename(columns={"close":f'{files_name[nums]}_close','volume':f'{files_name[nums]}_volume'},inplace=True) data1.set_index("time",inplace=True) data1 = data1[[f'{files_name[nums]}_close',f'{files_name[nums]}_volume']] #print(data1.head()) if bool(df_main.empty) != False: df_main = data1 else: df_main = df_main.join(data1) Same Code Just Some Alterations 😉

    Reply
  31. James Anderegg on

    Hey man, first off thanks for all you do! Really enjoy all your content. We don’t need to understand the minute by minute data… But I really would like to understand what the OPEN, and CLOSE is.. if it’s OPEN and CLOSE for every minute or HIGH and LOW for every minute why is the CLOSE not carried over to the next rows OPEN column..?

    Also, did you use a free service to gather the DATA or did you pay for it?

    Reply
  32. simon abdou on

    Just as a side note: Financial date usually come in OHLC format (open, high, low, close). Your data is probably the same but I could be wrong. I love your videos by the way..

    Reply
  33. Probability on

    Please feel free to do as many finance videos as you want! No complaints from me on this topic. Would be curious what other [sequential] data sets are being analyzed via RNNs…

    Reply
  34. Mathias Thorsen on

    My simple version to make the main_df:

    files = [“LTC-USD”, “BTC-USD”, “ETH-USD”, “BCH-USD”]
    main_list = []
    for file in files:
    df = pd.read_csv(f”crypto_data/{file}.csv”, names=[“open”, “low”, “high”, f”{file}_close”, f”{file}_volume”])
    main_list.append(df[[f”{file}_close”, f”{file}_volume”]])

    main_df = pd.concat(main_list, axis=1).dropna()

    Reply
  35. pennoo on

    How can I use the f (function) with dataset? The dataset in your code seems not defined ahead of the loop… just wondering.

    Reply
  36. KD2QMZ on

    Any idea of how this happened?:
    OSError: Unable to create file (unable to open file: name = ‘models/RNN_Final-01-0.559.model’, errno = 2, error message = ‘No such file or directory’, flags = 13, o_flags = 302) Thanks.

    Reply
  37. Hamish on

    probably not better, but it allows you to change the columns easily by dropping them using the list.

    names = [“time”, “low”, “high”, “open”, “close”, “volume”]
    ratios = [“BCH-USD”, “BTC-USD”, “ETH-USD”, “LTC-USD”]
    x = 0
    df_dict = {}
    for ratio in ratios:
    df = pd.read_csv(f”{ratio}.csv”, names = names).drop(columns = names[1:4])
    df_dict[f”dataframe{x}”] = df
    x += 1
    if x > len(ratios) – 1:
    pass
    else:
    df_dict[“dataframe0”] = pd.merge(df_dict[“dataframe0”], df, on=’time’, how=’left’, suffixes = (ratio, ratios[x]))
    main_df = df_dict[“dataframe0”]

    Reply
  38. Francesco Buccoliero on

    Find a Forex or something like that that offers api / build an api over that and let him play with a demo account. If it works try with a true one

    Reply
  39. Lun Zhang on

    Hello, hi. I hope to see some tutorials about using tensor flow to deal with video sequence (e.g. Like use CNN to do Human Action Recognition, because many open human action recognition dataset recorded human behavior as video). Thanks for your great tutorials.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Show Buttons
Hide Buttons