• 0 Posts
  • 117 Comments
Joined 1 year ago
cake
Cake day: June 11th, 2023

help-circle






  • I remember back in the day this automated downloader program… the links had a limit of one download at a time and you had to solve a captcha to start each download.

    So the downloader had built in “solve other’s captcha” system, where you could build up credit.

    So when you had say 20 links to download you spent some minutes solving other’s captchas and get some credit, then the program would use that crowdsourcing to solve yours as they popped up.














  • It’s less the calculations and more about memory bandwidth. To generate a token you need to go through all the model data, and that’s usually many many gigabytes. So the time it takes to read through in memory is usually longer than the compute time. GPUs have gb’s of RAM that’s many times faster than the CPU’s ram, which is the main reason it’s faster for llm’s.

    Most tpu’s don’t have much ram, and especially cheap ones.