Scraping Instagram Comments
You can legally export Instagram comments to Excel & CSV files from any public post without violating the Terms of Service using a little-known technique involving HAR files.
This allows you to download Instagram comments without Instagram even being aware of any scraping activity, as we’ll be scraping data from a recording of our web traffic instead of from the actual Instagram website. Watch the following video to learn more or read on to try for yourself.
1. Scroll Through Comments
Open up Instagram in your web browser, then navigate to the Instagram post URL you want to scrape the comments of. Click on the post and you’ll see the comments to the right of the post.
On your web browser (we’ll assume you’re using Google Chrome, but these instructions should be similar for any browser), right click anywhere on the page and hit Inspect to open up developer tools (this will allow you to record your web traffic as Instagram serves back the comments). You can then click on the “Network” tab to see your web traffic with the data in it.
You now want to refresh the page (now that you’re recording your web traffic), so you get the initial set of comments.
Move your mouse over the first few comments (these should be the most recent, or “top” comments depending on what scandal Instagram is trying to suppress at the moment), then scroll down to load more and more post comments. Your browser will be recording the raw data as you do this, so keep scrolling until you get to the end or you get tired of seeing spambot comments.
2. Export a HAR File
Once you’re done scrolling, look for a download arrow in your browser’s developer tools (Network Tab) that’s labeled “Export HAR…” & click this to save a HAR file containing the raw network data Instagram sent to your browser:
Now that we have the data inside this large file, the trick is to extract data out of this file. For this, we can use the HAR File Web Scraper where you can upload the file there, and the software will parse everything out for you and group together all the different requests with the comments in it:
3. Download Instagram Comments
You can click the disk icon next to each of the requests to download the data for free, or scroll up and click the “Parse Group” button (above all the individual responses) to download a combined CSV file with the comments in it:
You can see that the HAR File Scraper combined all of the individual requests into a combined list that you can now download as a single CSV file.
Check out the CSV file to see exactly what it contains, but you should get the full comment text, the name of the author, link to their profile pictures, and within the comment you’ll see Instagram hashtags used in the bodies, which can be fun to analyze! You can repeat this process as much as you’d like to export comments from several Instagram users & posts.
Why Scrape Instagram Comments?
Scraping Instagram comments can be helpful for better understanding how an audience is engagement with your content - or content from clients, competitors, or any other Instagram account you may be interested in.
Furthermore, you can see what hashtags are used in the comments and incorporate those hashtags into your future content. It can also be a good way to find the top comments for your overall Instagram page by scraping the comments for several posts and then ranking them all together by like count.
Legal Concerns
Despite how useful these comments can be for influencers, marketers & researchers, Instagram prohibits scraping comments via automated means, making collecting the raw comment data tricky, even from the best Instagram scrapers.
While there are automated browser extensions that promise to scrape Instagram data for you, these nearly all violate the Instagram Terms of Service, and using these unofficial tools while logged in to Instagram will often leave footprints behind that can result in your Instagram profile being blocked and/or banned.
There are other Instagram comment scrapers and web scrapers that do not run in your browser (safer since your account is not on the line), but they can be prohibitively expensive, relying on residential proxies and other web scraping “hacks” involved to avoid being detected and blocked. Furthermore, if you only need a few thousand or so comments, these data scraping tools are often overkill and usually not worth the effort needed to set up.