I am scraping a website using a proxy, that started showing captcha, I then scraped same website using requests.get() and that show just fine, whereas, at same time ...

None of responses by default are saved to disk. You must select for which requests the response HTMLs will be saved: def parse(self, response): """Processes start ...