This is fascinating! I had no idea that scrolling was such a complex and code-intensive function.
Loading more data as the user scrolls (or when they click a "More Results" button) is the easy part! The hard parts are almost everything else. 😉
- When the user scrolls through several pages of items, clicks a link to a different page, and then clicks the browser's back button, we need to take them back to exactly where they were in the list. Almost all websites get this wrong and just take you back to the first page as if you hadn't scrolled.
- When someone clicks on a link to a specific post in a conversation, we need to scroll them directly to that post and also load several posts before and after it. But we shouldn't try to load all posts before it, because it might be post 500 in a very long conversation.
- While loading posts before the post a user is looking at, we have to estimate the amount of space those posts will take up on the page so that the browser's scroll bar indicates roughly the right position even before they've been loaded. But we don't actually know how much space they'll take up yet because they haven't been loaded yet!
- When we render posts before the post a user is looking at, the difference between our estimate of their size and their actual size could cause the post the user is looking at to jump up or down on the page, so we have to quickly adjust the scroll position to compensate so that the user hopefully doesn't notice any change. This is surprisingly difficult.
- Search crawlers like Googlebot don't scroll like a human does when they visit a page to index it for a search engine, so we have to also support a non-infinite fallback mode to ensure that search bots can paginate by clicking links instead of scrolling. Otherwise they'd never see content beyond the first page.
- As the user scrolls down, we also need to unrender (or hide) the content they've scrolled past in order to avoid using up too much memory as we add more items to the page at the bottom. So at any given time, we're only rendering the content the user can actually see as they scroll. But this breaks the browser's "Find in this page" functionality if the user wants to search for something on the page, because the browser can only search content that's actually rendered, and we aren't rendering content the user can't see.
...and many, many more. 😅