Scrap Haveibeenpwned using Nodejs
by - Thursday, January 1, 1970 at 12:00 AM
use nodejs and puppeteer library[quote][color=#8f93a2][font=Consolas, 'Courier New', monospace][color=#c792ea]const[/color] [color=#f07178]puppeteer[/color] [color=#c792ea]=[/color] [color=#82aaff]require[/color][color=#8f93a2]([/color][color=#89ddff]"[/color][color=#c3e88d]puppeteer[/color][color=#89ddff]"[/color][color=#8f93a2])[/color][color=#89ddff];[/color][color=#c792ea]const[/color] [color=#f07178]fs[/color] [color=#c792ea]=[/color] [color=#82aaff]require[/color][color=#8f93a2]([/color][color=#89ddff]'[/color][color=#c3e88d]fs[/color][color=#89ddff]'[/color][color=#8f93a2])[/color][color=#89ddff];[/color][color=#c792ea]const[/color] [color=#f07178]userAgent[/color] [color=#c792ea]=[/color] [color=#89ddff]'[/color][color=#c3e88d]Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36[/color][color=#89ddff]'[/color][color=#89ddff];[/color][color=#c792ea]async[/color] [color=#c792ea]function[/color] [color=#82aaff]isPwned[/color][color=#89ddff]([/color][color=#ff5370]email[/color][color=#89ddff])[/color] [color=#89ddff]{[/color][color=#8f93a2] [/color][color=#c792ea]let[/color][color=#8f93a2] browser[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff][i]try[/i][/color] [color=#89ddff]{[/color][color=#8f93a2] browser [/color][color=#c792ea]=[/color] [color=#89ddff][i]await[/i][/color][color=#8f93a2] puppeteer[/color][color=#89ddff].[/color][color=#82aaff]launch[/color][color=#8f93a2]()[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#c792ea]const[/color] [color=#f07178]url[/color] [color=#c792ea]=[/color] [color=#89ddff]`[/color][color=#c3e88d]https://haveibeenpwned.com/unifiedsearch/[/color][color=#89ddff]${[/color][color=#8f93a2]email[/color][color=#89ddff]}[/color][color=#89ddff]`[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#c792ea]const[/color] [color=#89ddff][[/color][color=#f07178]page[/color][color=#89ddff]][/color] [color=#c792ea]=[/color] [color=#89ddff][i]await[/i][/color][color=#8f93a2] browser[/color][color=#89ddff].[/color][color=#82aaff]pages[/color][color=#8f93a2]()[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff][i]await[/i][/color][color=#8f93a2] page[/color][color=#89ddff].[/color][color=#82aaff]setUserAgent[/color][color=#8f93a2](userAgent)[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#c792ea]const[/color] [color=#f07178]response[/color] [color=#c792ea]=[/color] [color=#89ddff][i]await[/i][/color][color=#8f93a2] page[/color][color=#89ddff].[/color][color=#82aaff]goto[/color][color=#8f93a2](url)[/color][color=#89ddff];[/color][color=#8f93a2] console[/color][color=#89ddff].[/color][color=#82aaff]log[/color][color=#8f93a2]([/color][color=#89ddff][i]await[/i][/color][color=#8f93a2] response[/color][color=#89ddff].[/color][color=#82aaff]text[/color][color=#8f93a2]())[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#c792ea]const[/color] [color=#89ddff]{[/color] [color=#f07178]Breaches[/color] [color=#89ddff]}[/color] [color=#c792ea]=[/color] [color=#89ddff][i]await[/i][/color][color=#8f93a2] response[/color][color=#89ddff].[/color][color=#82aaff]json[/color][color=#8f93a2]()[/color][color=#89ddff];[/color][color=#8f93a2] console[/color][color=#89ddff].[/color][color=#82aaff]log[/color][color=#8f93a2](Breaches)[/color][color=#8f93a2] [/color][color=#89ddff][i]return[/i][/color][color=#8f93a2] Breaches[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff]}[/color] [color=#89ddff][i]catch[/i][/color][color=#8f93a2](error) [/color][color=#89ddff]{[/color][color=#8f93a2] console[/color][color=#89ddff].[/color][color=#82aaff]log[/color][color=#8f93a2](error)[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff][i]return[/i][/color][color=#8f93a2] [][/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff]}[/color] [color=#89ddff][i]finally[/i][/color] [color=#89ddff]{[/color][color=#8f93a2] [/color][color=#89ddff][i]if[/i][/color][color=#8f93a2] (browser) [/color][color=#89ddff]{[/color][color=#8f93a2] [/color][color=#89ddff][i]await[/i][/color][color=#8f93a2] browser[/color][color=#89ddff].[/color][color=#82aaff]close[/color][color=#8f93a2]()[/color][color=#89ddff];[/color][color=#8f93a2] [/color][color=#89ddff]}[/color][color=#8f93a2] [/color][color=#89ddff]}[/color][color=#89ddff]}[/color][color=#82aaff]isPwned[/color][color=#8f93a2]("[email protected]")[/color][color=#89ddff];[/color][/font][/color][/quote]it will return an array like this [quote][color=#8f93a2][font=Consolas, 'Courier New', monospace][color=#c792ea]var[/color] [color=#c792ea]=[/color] [color=#89ddff]{[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Breaches[/color][color=#89ddff]"[/color][color=#89ddff]:[/color][color=#8f93a2] [[/color][color=#8f93a2] [/color][color=#89ddff]{[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Name[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]Gravatar[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Title[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]Gravatar[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Domain[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]gravatar.com[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]BreachDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2020-10-03[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]AddedDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2021-12-05T22:45:58Z[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]ModifiedDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2021-12-08T01:47:02Z[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]PwnCount[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]113990759[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Description[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]In October 2020, a security researcher published a technique for scraping large volumes of data from Gravatar, the service for providing globally unique avatars . 167 million names, usernames and MD5 hashes of email addresses used to reference users' avatars were subsequently scraped and distributed within the hacking community. 114 million of the MD5 hashes were cracked and distributed alongside the source hash, thus disclosing the original email address and accompanying data. Following the impacted email addresses being searchable in HIBP, Gravatar release an FAQ detailing the incident.[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]LogoPath[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]https://haveibeenpwned.com/Content/Images/PwnedLogos/Gravatar.png[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]DataClasses[/color][color=#89ddff]"[/color][color=#89ddff]:[/color][color=#8f93a2] [[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Email addresses[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Names[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Usernames[/color][color=#89ddff]"[/color][color=#8f93a2] ][/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsVerified[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]true[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsFabricated[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsSensitive[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsRetired[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsSpamList[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsMalware[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#8f93a2] [/color][color=#89ddff]},[/color][color=#8f93a2] [/color][color=#89ddff]{[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Name[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]Tokopedia[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Title[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]Tokopedia[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Domain[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]tokopedia.com[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]BreachDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2020-04-17[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]AddedDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2020-05-02T23:45:21Z[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]ModifiedDate[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]2020-07-17T00:53:20Z[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]PwnCount[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]71443698[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Description[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]In April 2020, Indonesia's largest online store Tokopedia suffered a data breach. The incident resulted in 15M rows of data being posted to a popular hacking forum. An additional 76M rows were later provided to HIBP in July 2020. In total, the data included over 71M unique email addresses alongside names, genders, birth dates and passwords stored as SHA2-384 hashes.[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]LogoPath[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#89ddff]"[/color][color=#c3e88d]https://haveibeenpwned.com/Content/Images/PwnedLogos/Tokopedia.png[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]DataClasses[/color][color=#89ddff]"[/color][color=#89ddff]:[/color][color=#8f93a2] [[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Dates of birth[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Email addresses[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Genders[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Names[/color][color=#89ddff]"[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Passwords[/color][color=#89ddff]"[/color][color=#8f93a2] ][/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsVerified[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]true[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsFabricated[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsSensitive[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsRetired[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsSpamList[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]IsMalware[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]false[/color][color=#8f93a2] [/color][color=#89ddff]}[/color][color=#8f93a2] ][/color][color=#89ddff],[/color][color=#8f93a2] [/color][color=#89ddff]"[/color][color=#c3e88d]Pastes[/color][color=#89ddff]"[/color][color=#89ddff]:[/color] [color=#f78c6c]null[/color][color=#8f93a2] [/color][color=#89ddff]}[/color][/font][/color][/quote]why need puppeter ? because the API use cloudflare , we can't request to it just with http request
Sometimes we live like in the hell 


tg : t.me/insideheartz
wanna donate ? USDT : TTe5XaiadrL8kaPtB3tsyKonkqNjfwJw3S


Reply
heyy, puppeteer is my name bro
Reply
even if i prefer beautifulsoup and python thank you
Reply
(September 20, 2022, 04:45 PM)friteman007 Wrote: even if i prefer beautifulsoup and python thank you


You cannot use BeautifulSoup for this. You're using an API, not scraping the HTML.
Reply
(September 20, 2022, 05:24 PM)dwnlowd Wrote:
(September 20, 2022, 04:45 PM)friteman007 Wrote: even if i prefer beautifulsoup and python thank you


You cannot use BeautifulSoup for this. You're using an API, not scraping the HTML.


tru
Sometimes we live like in the hell 


tg : t.me/insideheartz
wanna donate ? USDT : TTe5XaiadrL8kaPtB3tsyKonkqNjfwJw3S


Reply
Nice,

Cloudflare Worker sometimes can be used for bypassing the verification, but I don't know what the limit is because I've never tried it in massive quantities
Reply
thanks
Reply


 Users viewing this thread: Scrap Haveibeenpwned using Nodejs: No users currently viewing.