This is not a programming question, the code is used to facilitate the demonstration of an issue in facebook app which can be reproduced manually without affecting the outcome.
Comment loading slows down exponentially with the number of comments loaded in browser. The code below uses selenium to login and gets this post which has 10k+ comments. Then the view more comments button is continuously clicked to load more comments as shown below:
from datetime import timedeltafrom time import perf_counterfrom selenium.common.exceptions import ( ElementClickInterceptedException, ElementNotInteractableException, StaleElementReferenceException,)from selenium.webdriver import Chrome, ChromeOptionsfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.support import expected_conditionsfrom selenium.webdriver.support.wait import WebDriverWaitdef login(driver, email, password): driver.find_element(By.ID, 'email').send_keys(email) driver.find_element(By.ID, 'pass').send_keys(password) driver.find_element(By.XPATH, '//*[contains(@type, "submit")]').click() WebDriverWait(driver, 1000).until( expected_conditions.presence_of_element_located( (By.XPATH, '//*[contains(@action, "/logout")]') ) )if __name__ == '__main__': options = ChromeOptions() options.add_argument('disable-notifications') _driver = Chrome(options=options) _driver.get('https://facebook.com') login(_driver, 'email', 'password') # replace with credentials _driver.get('https://www.facebook.com/SarcasmSociety/photos/a.10150919189073866/10155498125313866' ) reload_btn = WebDriverWait(_driver, 1000).until( expected_conditions.presence_of_element_located( (By.XPATH, '//*[contains(text(), "View more comments")]') ) ) prev_count = 0 t = perf_counter() while True: _driver.execute_script('arguments[0].scrollIntoView();', reload_btn) try: reload_btn.click() except ( ElementClickInterceptedException, StaleElementReferenceException, ElementNotInteractableException, ): pass total_loaded = ( WebDriverWait(_driver, 1000) .until( expected_conditions.presence_of_element_located( (By.XPATH, '//div[4]/div[2]/div[2]/span') ) ) .text.split('of')[0] ) if prev_count != total_loaded: print( f'comments loaded: {total_loaded} load time {timedelta(seconds=perf_counter() - t)}' ) prev_count = total_loaded t = perf_counter()As the number of comments loaded increases, the loading time keeps increasing and the browser eventually hangs and crashes. I tried with chrome, safari, firefox and I get the same result.
comments loaded: 6 load time 0:00:00.956970comments loaded: 56 load time 0:00:05.018559comments loaded: 106 load time 0:00:05.097524comments loaded: 156 load time 0:00:12.881246comments loaded: 206 load time 0:00:14.915316comments loaded: 256 load time 0:00:07.909231comments loaded: 306 load time 0:00:14.662676comments loaded: 356 load time 0:00:17.050463comments loaded: 406 load time 0:00:14.737875comments loaded: 456 load time 0:00:10.441015comments loaded: 506 load time 0:00:22.218294comments loaded: 556 load time 0:00:20.039933comments loaded: 606 load time 0:00:14.072164comments loaded: 656 load time 0:00:33.570592comments loaded: 706 load time 0:00:29.157080comments loaded: 756 load time 0:00:27.104434comments loaded: 806 load time 0:00:28.285868comments loaded: 856 load time 0:00:29.938833comments loaded: 906 load time 0:00:31.281724comments loaded: 956 load time 0:00:36.025857comments loaded: 1,006 load time 0:00:34.422590comments loaded: 1,056 load time 0:00:32.973732comments loaded: 1,156 load time 0:01:16.370328comments loaded: 1,206 load time 0:00:44.005518comments loaded: 1,256 load time 0:00:47.166408comments loaded: 1,306 load time 0:00:44.392992comments loaded: 1,356 load time 0:01:01.694369comments loaded: 1,406 load time 0:00:28.100825comments loaded: 1,456 load time 0:00:52.131026comments loaded: 1,506 load time 0:00:57.448965comments loaded: 1,556 load time 0:00:53.571871comments loaded: 1,606 load time 0:00:57.142948comments loaded: 1,656 load time 0:00:58.251598comments loaded: 1,706 load time 0:01:33.050878comments loaded: 1,756 load time 0:00:33.725328comments loaded: 1,856 load time 0:02:20.364518comments loaded: 1,906 load time 0:01:08.370199comments loaded: 1,956 load time 0:01:22.470486comments loaded: 2,006 load time 0:01:35.449901comments loaded: 2,056 load time 0:02:20.828311comments loaded: 2,106 load time 0:00:49.856041comments loaded: 2,156 load time 0:01:38.646713comments loaded: 2,206 load time 0:01:38.485394comments loaded: 2,256 load time 0:01:46.626105comments loaded: 2,306 load time 0:02:00.718524Traceback (most recent call last): File "/Users/user/Library/Application Support/JetBrains/PyCharm2022.3/scratches/scratch_2.py", line 48, in <module> reload_btn.click() File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 93, in click self._execute(Command.CLICK_ELEMENT) File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webelement.py", line 410, in _execute return self._parent.execute(command, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 444, in execute self.error_handler.check_response(response) File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 249, in check_response raise exception_class(message, screen, stacktrace)selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crashfrom unknown error: cannot determine loading statusfrom tab crashed (Session info: chrome=108.0.5359.124)Stacktrace:0 chromedriver 0x0000000100d16f38 chromedriver + 49109041 chromedriver 0x0000000100c96a03 chromedriver + 43852832 chromedriver 0x00000001008db5bd chromedriver + 4725093 chromedriver 0x00000001008c4064 chromedriver + 3769324 chromedriver 0x00000001008c27f2 chromedriver + 3706745 chromedriver 0x00000001008c2d18 chromedriver + 3719926 chromedriver 0x00000001008d144c chromedriver + 4311807 chromedriver 0x00000001008d2752 chromedriver + 4360508 chromedriver 0x00000001008e50ce chromedriver + 5122069 chromedriver 0x0000000100961425 chromedriver + 102096510 chromedriver 0x0000000100945ee3 chromedriver + 90902711 chromedriver 0x000000010091030c chromedriver + 68890812 chromedriver 0x000000010091188e chromedriver + 69441413 chromedriver 0x0000000100ce41de chromedriver + 470268614 chromedriver 0x0000000100ce8b19 chromedriver + 472143315 chromedriver 0x0000000100cf028e chromedriver + 475201416 chromedriver 0x0000000100ce991a chromedriver + 472501817 chromedriver 0x0000000100cbdb02 chromedriver + 454528218 chromedriver 0x0000000100d08888 chromedriver + 485184819 chromedriver 0x0000000100d08a05 chromedriver + 485222920 chromedriver 0x0000000100d1ee5f chromedriver + 494345521 libsystem_pthread.dylib 0x00007ff8163df259 _pthread_start + 12522 libsystem_pthread.dylib 0x00007ff8163dac7b thread_start + 15python-BaseExceptionAt first I thought the browser is being crowded by comments and that's why it's crashing and maybe deletion of the loaded comments from the page might fix the issue, so I tried:
for (let element of $x("//div[4]/ul/li")) { element.parentNode.removeChild(element)}Nothing improves and the browser eventually crashes again. I thought perhaps the reload requests are being throttled by the server specially after getting the same exact response time multiple times. But if this is the case, the page won't freeze and stop responding, just the delay would be expected in this case without a browser crash. So, what's causing the crash? How to avoid this outcome and eventually load all comments in a reasonable time?