Unfortunatly, (as much as I would love to), I do not have the neccesary resources to achieve a similar solution. (Some people actually have went through the trouble of using machine learning to solve captchas). Now I've seen many different solutions to the CAPTCHA problem. These tests usually include identifying common objects or typing letters that are played through speakers. This was bad news, a CAPTCHA is essentially a test web developers make for their users to make sure that the user is human and not a robot stealing their data. As it turns out Tiktok doesn't like it when you pull a lot of data from them, so if you send a lot of get requests to their web servers, you will be profiled as a robot and asked to complete a CAPTCHA. When I first started coding this bot I was faced with a big problem. I plan to continue updating this as I keep adding more and more features, and learn more about how websites handle data The CAPTCHA Problem I then started doing research and came to the realization that it may be possible for me to try and make one. I had already heard about how it's possible to make API's for websites using a term what's known as web-scraping, however I didn't have any knowledge on the matter. Initially, my plan was to write a simple app using the official Tiktok API that just checked the follower count for each of my followers but I soon came to realize that Tiktok had no official API I decided to work on this project mainly for myself, because I wanted to see who the most popular person that followed me on Tiktok was. This should be enough to get you started on incorporating this API into whatever programs you wish to make.įull documentation available here Inspiration Go to the terminal and type in the following: Since there are a multitude of package installers for Linux, we'll choose apt which is used in Debian-derived distros such as Ubuntu as an example. Once you have opened the installer, make sure that you add Python 3.8 to path. If you don't have these installed please read the guide below Installing Python3 for WindowsĬlick on the following link and open it once it's downloaded: To check if you have these installed run python -V (make sure the version is python 3.X.X) and python -m pip -V. Table of ContentsĪs usual, first you need to make sure you have Python 3 + pip installed. However Tiktok is a $250 Billion company and I am a 16 year old with a laptop. I do agree that mass spamming requests with hundreds of IPs in an attempt to DDOS, or even just in an attempt to retreive a huge amount of data from a company's servers (especially if the company/company's servers are smaller than typical big corperation/big corperation's servers) is a blatant violation of said moral codes.
#Write your own webscraper code#
It is my belief that due to all of the data being scraped in this project's current publicly accesible state, writing a program that goes through the frontend code to scrape data goes against no moral codes. I do want to preface one thing first though, I have talked to many developers about the morality behind web scraping, and learned many different opinions on if ethically it should be done. The problem is humans love data, which means we will stop at nothing to find ways to make data more accessible to us. Web scraping is a really important tool to learn, even if it's generally fround apon by companies.
#Write your own webscraper download#
I don't expect anyone to download this but I'm still going to write a fully fleshed out README with usage tutorials and everything for completion's sake This project was started, mainly as a way for me to practice my webscraping skills and class structure knowledge. follower count, like count, etc.) Overview Simple webscraping bot that has a set of methods to get specific and general information about a specified Tiktok user/video (e.g.
0 kommentar(er)
