BrowserAPI Scroll Methods
The BrowserAPI class provides five scroll methods through the
session.browser interface that give you precise control over page
scrolling within an adaptive mode BBRE session. The scroll(options)
method is the generic scroll action that accepts an options object for maximum
flexibility, allowing you to pass custom scroll parameters directly to the BBRE
engine. The scrollDown(pixels) method scrolls the page downward by a
specified number of pixels, defaulting to 500 pixels when no value is provided.
The scrollUp(pixels) method scrolls the page upward by a specified
number of pixels, also defaulting to 500 pixels. The scrollToTop()
method instantly scrolls to the very top of the page, and the
scrollToBottom() method instantly scrolls to the very bottom. Together,
these five methods cover every scrolling scenario you will encounter in browser
automation, from incrementally loading infinite scroll feeds and triggering lazy
image loading to revealing content hidden below the fold and navigating back to the
top of long pages. Scrolling is a fundamental part of modern web interaction because
many websites defer content loading until the user scrolls to a specific region of
the page. Without proper scroll control, your automation scripts will miss
dynamically loaded products, images, comments, and other content that only appears
after scrolling. This page provides complete documentation for every scroll method,
including method signatures, parameter tables, return types, practical code examples
covering infinite scroll harvesting, lazy image triggering, progressive content
loading, scroll position verification, and combined scroll-and-wait patterns, along
with best practices, common issues, and links to related documentation.
All BrowserAPI scroll methods require an active session running in
adaptive mode. Adaptive mode sessions launch a real browser instance
managed by the BBRE engine, which is necessary for executing scroll actions against
the live viewport. If you attempt to call scroll methods on a passive mode session,
the operation will fail with a BROWSER_ACTION_FAILED error. Create your
session with mode: "adaptive" to use these methods. For simple HTTP
requests where you only need the raw HTML response without browser rendering or
scroll-dependent content, consider using passive mode with
session.request() instead, which is faster and consumes fewer credits.
Modern websites heavily rely on scroll-triggered behaviors to optimize performance and user experience. Infinite scroll feeds load new batches of content as the user reaches the bottom of the current viewport. Lazy loading defers image and media downloads until those elements scroll into view. Parallax effects, sticky headers, and scroll-based animations all depend on the scroll position to function correctly. When automating interactions with these websites, your script must replicate the scrolling behavior that a real user would perform. Without scrolling, the browser DOM will only contain the initially visible content, and any elements that depend on scroll events will remain unloaded, invisible, or in their default state. The BBRE scroll methods send real scroll events to the browser engine, triggering the same JavaScript event handlers and intersection observers that fire during manual user scrolling.
session.browser.scroll(options)
The scroll() method is the generic scroll action that sends a scroll
command to the BBRE engine with a custom options object. This method provides the
most flexibility among the scroll methods because you can pass any combination of
scroll parameters supported by the engine. When called without arguments, it sends
an empty options object, which the engine interprets as a default scroll action.
The scroll() method is the foundation that the other four convenience
methods build upon internally. Use this method when you need to pass custom scroll
parameters that are not covered by the convenience methods, or when you want to
forward a dynamically constructed options object to the scroll action.
Method Signature
await session.browser.scroll(options);
Parameters
The scroll() method accepts a single optional parameter: an options
object that is forwarded directly to the BBRE engine's scroll action handler. If
omitted, an empty object is sent by default.
| Parameter | Type | Required | Description |
|---|---|---|---|
options |
object | Optional | An object containing scroll parameters to forward to the BBRE engine. The supported properties depend on the engine implementation. When omitted, an empty object is sent as the default. |
Return Type
The method returns a Promise that resolves to the action result object
from the BBRE engine. The result contains information about the scroll action
execution status.
| Return Type | Description |
|---|---|
| Promise<object> | Resolves to the action result from the BBRE engine containing the scroll execution status and any additional data returned by the engine. |
Basic Usage
The simplest usage of scroll() sends a default scroll action to the
engine without any custom parameters. This is useful when you want to trigger a
generic scroll event on the page.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function genericScroll() {
await session.start();
await session.browser.navigate("https://example.com/long-page");
await session.browser.scroll();
const pageContent = await session.browser.getText("body");
console.log("Page content length after scroll:", pageContent.length);
await session.close();
}
genericScroll();
Scroll with Custom Options
Pass a custom options object to control the scroll behavior. The options are forwarded directly to the BBRE engine, giving you access to any scroll parameters the engine supports.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollWithOptions() {
await session.start();
await session.browser.navigate("https://example.com/catalog");
await session.browser.scroll({ direction: "down", amount: 800 });
await session.browser.wait(1);
await session.browser.scroll({ direction: "up", amount: 200 });
await session.close();
}
scrollWithOptions();
session.browser.scrollDown(pixels)
The scrollDown() method scrolls the page downward by a specified number
of pixels. It sends a scrollDown action to the BBRE engine with the
pixel value wrapped in a { pixels } object. When called without an
argument, the method defaults to scrolling down by 500 pixels, which is roughly
equivalent to one viewport height on most desktop screens. This method is the most
frequently used scroll method in automation workflows because the majority of
scroll-dependent content loading happens when the user scrolls downward. Use
scrollDown() to incrementally load infinite scroll feeds, trigger lazy
image loading for elements below the current viewport, reveal content that is hidden
below the fold, and simulate natural user browsing behavior where the user reads
content from top to bottom. The pixel value gives you precise control over how far
the page scrolls with each call, allowing you to fine-tune the scroll distance based
on the target website's content density and loading triggers.
Method Signature
await session.browser.scrollDown(pixels);
Parameters
The scrollDown() method accepts a single optional parameter specifying
the number of pixels to scroll downward. If omitted, the method defaults to 500
pixels.
| Parameter | Type | Required | Description |
|---|---|---|---|
pixels |
number | Optional | The number of pixels to scroll downward. Must be a positive number. Defaults to 500 when omitted. Larger values scroll further down the page in a single action. |
Return Type
The method returns a Promise that resolves to the action result object
from the BBRE engine confirming the scroll action was executed.
| Return Type | Description |
|---|---|
| Promise<object> | Resolves to the action result from the BBRE engine containing the scroll execution status. |
Basic Usage
The simplest usage of scrollDown() scrolls the page down by the default
500 pixels. This is enough to move past the initial viewport on most pages and
trigger any content loading that depends on the user scrolling past the fold.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollPageDown() {
await session.start();
await session.browser.navigate("https://example.com/products");
await session.browser.scrollDown();
await session.browser.wait(1);
const visibleProducts = await session.browser.getText(".product-list");
console.log("Products visible after scroll:", visibleProducts.length);
await session.close();
}
scrollPageDown();
Scroll Down with Custom Pixel Value
Pass a custom pixel value to control exactly how far the page scrolls. Smaller values like 200 or 300 pixels create a gentle scroll that mimics careful reading behavior. Larger values like 1000 or 2000 pixels jump further down the page, which is useful when you need to reach content deep in the page quickly.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function customScrollDown() {
await session.start();
await session.browser.navigate("https://example.com/article");
await session.browser.scrollDown(300);
await session.browser.wait(0.5);
await session.browser.scrollDown(300);
await session.browser.wait(0.5);
await session.browser.scrollDown(1500);
await session.browser.wait(1);
const footerText = await session.browser.getText("footer");
console.log("Footer content:", footerText);
await session.close();
}
customScrollDown();
Incremental Scroll to Load Lazy Images
Many websites use lazy loading to defer image downloads until the image element scrolls into the viewport. To ensure all images on a page are loaded, scroll down incrementally in small steps, pausing briefly after each scroll to give the browser time to detect the newly visible images and start downloading them. This pattern is essential for scraping product catalogs, image galleries, and media-heavy pages where the initial page load only includes placeholder images.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function loadAllLazyImages() {
await session.start();
await session.browser.navigate("https://example.com/gallery");
await session.browser.wait(2);
const scrollSteps = 15;
const pixelsPerStep = 400;
for (let i = 0; i < scrollSteps; i++) {
await session.browser.scrollDown(pixelsPerStep);
await session.browser.wait(0.8);
}
const pageHtml = await session.browser.getHtml("body");
const imageCount = (pageHtml.match(/<img /g) || []).length;
console.log("Total images found after scrolling:", imageCount);
await session.close();
}
loadAllLazyImages();
Error Handling for scrollDown()
The scrollDown() method throws an error when the session is not in
adaptive mode, the session has expired or been closed, or the BBRE engine encounters
an internal error while executing the scroll action. Always wrap scroll calls in a
try-catch block when building production automation workflows.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function safeScrollDown(pixels) {
try {
await session.browser.scrollDown(pixels);
console.log("Scrolled down", pixels || 500, "pixels");
return true;
} catch (error) {
console.log("Scroll failed:", error.message);
return false;
}
}
async function main() {
await session.start();
await session.browser.navigate("https://example.com/page");
const success = await safeScrollDown(600);
if (success) {
const content = await session.browser.getText("body");
console.log("Content length:", content.length);
}
await session.close();
}
main();
session.browser.scrollUp(pixels)
The scrollUp() method scrolls the page upward by a specified number of
pixels. It sends a scrollUp action to the BBRE engine with the pixel
value wrapped in a { pixels } object. When called without an argument,
the method defaults to scrolling up by 500 pixels. This method is the counterpart
to scrollDown() and is useful in scenarios where you need to navigate
back to previously viewed content, interact with elements that have scrolled out of
the viewport above the current position, or implement bidirectional scroll patterns
that simulate natural user behavior. A common use case is scrolling down to load
content, then scrolling back up to interact with elements that appeared during the
downward scroll. Another scenario is verifying that sticky headers or floating
navigation bars behave correctly at different scroll positions. The pixel value
controls how far the page scrolls upward with each call, giving you the same
precision as scrollDown() but in the opposite direction.
Method Signature
await session.browser.scrollUp(pixels);
Parameters
The scrollUp() method accepts a single optional parameter specifying
the number of pixels to scroll upward. If omitted, the method defaults to 500
pixels.
| Parameter | Type | Required | Description |
|---|---|---|---|
pixels |
number | Optional | The number of pixels to scroll upward. Must be a positive number. Defaults to 500 when omitted. Larger values scroll further up the page in a single action. |
Return Type
The method returns a Promise that resolves to the action result object
from the BBRE engine confirming the scroll action was executed.
| Return Type | Description |
|---|---|
| Promise<object> | Resolves to the action result from the BBRE engine containing the scroll execution status. |
Basic Usage
The simplest usage of scrollUp() scrolls the page up by the default
500 pixels. This is useful after scrolling down to load content, when you need to
return to a section that is now above the current viewport.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollBackUp() {
await session.start();
await session.browser.navigate("https://example.com/long-article");
await session.browser.scrollDown(1500);
await session.browser.wait(1);
await session.browser.scrollUp();
await session.browser.wait(0.5);
const headerText = await session.browser.getText("h1");
console.log("Header after scrolling back up:", headerText);
await session.close();
}
scrollBackUp();
Scroll Up with Custom Pixel Value
Pass a custom pixel value to control exactly how far the page scrolls upward. This is useful when you need to scroll back to a specific section of the page that you know is a certain distance above the current scroll position.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function preciseScrollUp() {
await session.start();
await session.browser.navigate("https://example.com/search-results");
await session.browser.scrollDown(2000);
await session.browser.wait(1);
const resultText = await session.browser.getText(".result-item:last-child");
console.log("Last visible result:", resultText);
await session.browser.scrollUp(800);
await session.browser.wait(0.5);
const filterText = await session.browser.getText(".filter-panel");
console.log("Filter panel content:", filterText);
await session.close();
}
preciseScrollUp();
Bidirectional Scroll Pattern
Combine scrollDown() and scrollUp() to create a
bidirectional scroll pattern that mimics natural user behavior. Real users do not
scroll in a single direction continuously. They scroll down to browse, scroll back
up to re-read something, then continue scrolling down. This pattern is useful for
making your automation appear more human-like and for interacting with content at
different positions on the page.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function humanLikeScrolling() {
await session.start();
await session.browser.navigate("https://example.com/blog-post");
await session.browser.scrollDown(600);
await session.browser.wait(2);
await session.browser.scrollDown(400);
await session.browser.wait(1.5);
await session.browser.scrollUp(200);
await session.browser.wait(1);
await session.browser.scrollDown(800);
await session.browser.wait(2);
await session.browser.scrollDown(500);
await session.browser.wait(1);
await session.browser.scrollUp(300);
await session.browser.wait(0.5);
await session.browser.scrollDown(1000);
await session.browser.wait(1);
const pageContent = await session.browser.getText("body");
console.log("Total content length:", pageContent.length);
await session.close();
}
humanLikeScrolling();
session.browser.scrollToTop()
The scrollToTop() method instantly scrolls the page to the very top,
setting the scroll position to zero regardless of the current scroll position. It
sends a scrollToTop action to the BBRE engine with an empty options
object. This method does not accept any parameters because its behavior is absolute
rather than relative. No matter how far down the page you have scrolled, calling
scrollToTop() will always bring you back to the beginning of the page.
This method is useful when you need to return to the top of the page after
completing a scroll-based content loading sequence, when you want to interact with
header elements, navigation menus, or search bars that are positioned at the top of
the page, or when you need to reset the scroll position before starting a new
interaction sequence on the same page. Unlike calling scrollUp() with
a large pixel value, scrollToTop() guarantees that you reach the
absolute top of the page regardless of the current scroll depth.
Method Signature
await session.browser.scrollToTop();
Parameters
The scrollToTop() method does not accept any parameters. It always
scrolls to the absolute top of the page.
| Parameter | Type | Required | Description |
|---|---|---|---|
| This method does not accept any parameters. | |||
Return Type
The method returns a Promise that resolves to the action result object
from the BBRE engine confirming the scroll-to-top action was executed.
| Return Type | Description |
|---|---|
| Promise<object> | Resolves to the action result from the BBRE engine confirming the page has been scrolled to the top. |
Basic Usage
The simplest usage of scrollToTop() returns to the top of the page
after scrolling down. This is useful when you need to access header navigation,
search bars, or other elements that are fixed at the top of the page layout.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function returnToTop() {
await session.start();
await session.browser.navigate("https://example.com/products");
for (let i = 0; i < 10; i++) {
await session.browser.scrollDown(500);
await session.browser.wait(0.5);
}
console.log("Finished loading content, returning to top");
await session.browser.scrollToTop();
await session.browser.wait(0.5);
await session.browser.click(".search-input");
await session.browser.fill(".search-input", "wireless headphones");
await session.browser.click(".search-button");
console.log("Search submitted from top of page");
await session.close();
}
returnToTop();
Reset Scroll Position Between Page Sections
When your automation workflow processes multiple sections of the same page, use
scrollToTop() to reset the scroll position between sections. This
ensures that each section processing starts from a known position, making your
scroll calculations predictable and reliable.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function processPageSections() {
await session.start();
await session.browser.navigate("https://example.com/dashboard");
await session.browser.wait(2);
const headerData = await session.browser.getText(".dashboard-header");
console.log("Header section:", headerData);
await session.browser.scrollDown(800);
await session.browser.wait(1);
const statsData = await session.browser.getText(".statistics-panel");
console.log("Statistics section:", statsData);
await session.browser.scrollToTop();
await session.browser.wait(0.5);
await session.browser.scrollDown(1600);
await session.browser.wait(1);
const activityData = await session.browser.getText(".recent-activity");
console.log("Activity section:", activityData);
await session.browser.scrollToTop();
console.log("Scroll position reset to top");
await session.close();
}
processPageSections();
session.browser.scrollToBottom()
The scrollToBottom() method instantly scrolls the page to the very
bottom, jumping to the maximum scroll position regardless of the current scroll
position. It sends a scrollToBottom action to the BBRE engine with an
empty options object. Like scrollToTop(), this method does not accept
any parameters because its behavior is absolute. Calling
scrollToBottom() will always bring you to the end of the page content.
This method is useful when you need to access footer elements, copyright notices,
or links that are positioned at the bottom of the page, when you want to quickly
check the total page length by scrolling to the end, or when you need to trigger
all lazy-loaded content at once by jumping to the bottom. However, be aware that
jumping directly to the bottom may not trigger all intermediate scroll events that
some websites rely on for progressive content loading. If a website uses infinite
scroll that loads content in batches as you scroll down, jumping to the bottom will
only trigger the scroll event at the final position, potentially missing the
intermediate loading triggers. In such cases, use incremental
scrollDown() calls instead to ensure all content batches are loaded.
Method Signature
await session.browser.scrollToBottom();
Parameters
The scrollToBottom() method does not accept any parameters. It always
scrolls to the absolute bottom of the page.
| Parameter | Type | Required | Description |
|---|---|---|---|
| This method does not accept any parameters. | |||
Return Type
The method returns a Promise that resolves to the action result object
from the BBRE engine confirming the scroll-to-bottom action was executed.
| Return Type | Description |
|---|---|
| Promise<object> | Resolves to the action result from the BBRE engine confirming the page has been scrolled to the bottom. |
Basic Usage
The simplest usage of scrollToBottom() jumps directly to the bottom of
the page. This is useful when you need to access footer content, load-more buttons
at the bottom, or any elements that are positioned at the end of the page.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function jumpToBottom() {
await session.start();
await session.browser.navigate("https://example.com/terms-of-service");
await session.browser.scrollToBottom();
await session.browser.wait(1);
const footerLinks = await session.browser.getText("footer");
console.log("Footer content:", footerLinks);
await session.close();
}
jumpToBottom();
Accept Terms by Scrolling to Bottom
Some websites require users to scroll to the bottom of a terms of service or
license agreement before enabling an accept button. Use
scrollToBottom() to reach the end of the agreement, then interact with
the accept button that becomes enabled after the full document has been viewed.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function acceptTermsOfService() {
await session.start();
await session.browser.navigate("https://example.com/signup");
await session.browser.fill("#email", "[email protected]");
await session.browser.fill("#password", "securePassword123");
await session.browser.scrollToBottom();
await session.browser.wait(1);
await session.browser.checkbox("#agree-terms", true);
await session.browser.click("#submit-button");
await session.browser.wait(2);
const confirmationText = await session.browser.getText(".confirmation-message");
console.log("Confirmation:", confirmationText);
await session.close();
}
acceptTermsOfService();
Scroll to Bottom and Back to Top
Combine scrollToBottom() and scrollToTop() to quickly
traverse the entire page. This pattern is useful for triggering all scroll-based
events on the page in a single pass, ensuring that any content that loads on scroll
is fully rendered before you start extracting data.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function fullPageTraversal() {
await session.start();
await session.browser.navigate("https://example.com/product-page");
await session.browser.wait(2);
await session.browser.scrollToBottom();
await session.browser.wait(2);
await session.browser.scrollToTop();
await session.browser.wait(1);
const fullPageHtml = await session.browser.getHtml("body");
console.log("Full page HTML length:", fullPageHtml.length);
const title = await session.browser.getText("h1");
console.log("Product title:", title);
await session.close();
}
fullPageTraversal();
Infinite Scroll Loading Pattern
Infinite scroll is one of the most common content loading patterns on modern websites. Instead of paginating content across multiple pages, websites load new batches of items as the user scrolls toward the bottom of the current content. Social media feeds, product catalogs, news aggregators, and search results frequently use this pattern. To scrape all available content from an infinite scroll page, you need to repeatedly scroll down, wait for new content to load, and check whether the page has reached its end. The following examples demonstrate different approaches to handling infinite scroll, from simple fixed-iteration scrolling to intelligent content-change detection.
Fixed Iteration Infinite Scroll
The simplest approach to infinite scroll loading is to scroll down a fixed number of times, pausing after each scroll to allow new content to load. This approach works well when you know approximately how many scroll iterations are needed to load all the content you want, or when you only need a specific number of items rather than the complete dataset.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function fixedIterationScroll() {
await session.start();
await session.browser.navigate("https://example.com/feed");
await session.browser.wait(3);
const maxScrolls = 20;
const scrollPixels = 800;
const waitBetweenScrolls = 1.5;
for (let i = 0; i < maxScrolls; i++) {
await session.browser.scrollDown(scrollPixels);
await session.browser.wait(waitBetweenScrolls);
console.log("Scroll iteration", i + 1, "of", maxScrolls);
}
const allItems = await session.browser.getText(".feed-item");
console.log("Total items loaded:", allItems.length);
await session.close();
}
fixedIterationScroll();
Content-Change Detection Infinite Scroll
A more intelligent approach is to detect when new content has stopped loading by comparing the page content before and after each scroll. When the content length stops changing between scroll iterations, you know that either all content has been loaded or the page has reached its end. This approach adapts to pages of any length without requiring you to guess the number of scroll iterations.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function smartInfiniteScroll() {
await session.start();
await session.browser.navigate("https://example.com/products");
await session.browser.wait(3);
let previousContentLength = 0;
let unchangedCount = 0;
const maxUnchanged = 3;
const maxScrollAttempts = 50;
for (let i = 0; i < maxScrollAttempts; i++) {
await session.browser.scrollDown(600);
await session.browser.wait(2);
const currentContent = await session.browser.getText("body");
const currentLength = currentContent.length;
if (currentLength === previousContentLength) {
unchangedCount++;
console.log("No new content detected. Unchanged count:", unchangedCount);
if (unchangedCount >= maxUnchanged) {
console.log("Content stopped loading after", i + 1, "scrolls");
break;
}
} else {
unchangedCount = 0;
console.log("New content loaded. Length:", previousContentLength, "->", currentLength);
}
previousContentLength = currentLength;
}
const finalContent = await session.browser.getHtml("body");
console.log("Final page HTML length:", finalContent.length);
await session.close();
}
smartInfiniteScroll();
Infinite Scroll with Item Count Target
When you need a specific number of items from an infinite scroll page, combine scrolling with item counting. After each scroll, count the number of items currently loaded and stop scrolling once you have reached your target count. This approach is efficient because it avoids unnecessary scrolling beyond what you actually need.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollUntilItemCount(targetCount) {
await session.start();
await session.browser.navigate("https://example.com/listings");
await session.browser.wait(3);
const maxAttempts = 40;
for (let i = 0; i < maxAttempts; i++) {
const itemsHtml = await session.browser.getHtml(".listing-grid");
const currentCount = (itemsHtml.match(/class="listing-card"/g) || []).length;
console.log("Current items:", currentCount, "Target:", targetCount);
if (currentCount >= targetCount) {
console.log("Target reached after", i + 1, "scrolls");
break;
}
await session.browser.scrollDown(700);
await session.browser.wait(1.5);
}
const finalHtml = await session.browser.getHtml(".listing-grid");
console.log("Final HTML length:", finalHtml.length);
await session.close();
}
scrollUntilItemCount(100);
Lazy Image Loading Pattern
Lazy loading is a performance optimization technique where images and other media
elements are not downloaded until they scroll into or near the viewport. Websites
implement lazy loading using the native loading="lazy" attribute,
Intersection Observer API, or custom JavaScript scroll event handlers. When
automating interactions with pages that use lazy loading, you need to scroll through
the entire page to trigger the loading of all images. Without scrolling, the page
HTML will contain placeholder elements or low-resolution thumbnails instead of the
actual image URLs. The following examples demonstrate how to use the scroll methods
to ensure all lazy-loaded images are fully loaded before extracting image data.
Progressive Scroll for Image Loading
Scroll through the page in small increments, pausing after each scroll to give the browser time to detect newly visible images and start downloading them. The pause duration should be long enough for the browser to fire intersection observer callbacks and initiate image downloads, but short enough to keep the total scroll time reasonable.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function loadAllImages() {
await session.start();
await session.browser.navigate("https://example.com/photo-gallery");
await session.browser.wait(3);
const viewportHeight = 400;
const totalScrolls = 25;
for (let i = 0; i < totalScrolls; i++) {
await session.browser.scrollDown(viewportHeight);
await session.browser.wait(1);
}
await session.browser.scrollToTop();
await session.browser.wait(2);
const galleryHtml = await session.browser.getHtml(".gallery-container");
console.log("Gallery HTML length:", galleryHtml.length);
await session.close();
}
loadAllImages();
Scroll and Verify Image Sources
After scrolling to trigger lazy loading, verify that the images have actually loaded
by checking their src attributes. Lazy-loaded images typically start
with a placeholder src (such as a data URI or a tiny placeholder image)
and get replaced with the real image URL once they scroll into view. By checking the
src values after scrolling, you can confirm that the lazy loading has
completed successfully.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function verifyLazyLoadedImages() {
await session.start();
await session.browser.navigate("https://example.com/product-catalog");
await session.browser.wait(3);
const beforeScrollHtml = await session.browser.getHtml(".product-grid");
const placeholderCount = (beforeScrollHtml.match(/data:image/g) || []).length;
console.log("Placeholder images before scroll:", placeholderCount);
for (let i = 0; i < 20; i++) {
await session.browser.scrollDown(500);
await session.browser.wait(0.8);
}
await session.browser.scrollToTop();
await session.browser.wait(2);
const afterScrollHtml = await session.browser.getHtml(".product-grid");
const remainingPlaceholders = (afterScrollHtml.match(/data:image/g) || []).length;
console.log("Placeholder images after scroll:", remainingPlaceholders);
console.log("Images loaded:", placeholderCount - remainingPlaceholders);
await session.close();
}
verifyLazyLoadedImages();
Scroll to Reveal Content Pattern
Many websites use scroll-triggered animations and visibility changes to create engaging user experiences. Elements may be hidden, faded out, or positioned off-screen until the user scrolls to their location, at which point they animate into view. This pattern is common for marketing pages, landing pages, and portfolio websites. When automating interactions with these pages, you need to scroll to the target element's position to trigger its reveal animation before you can interact with it or extract its content. The following examples demonstrate how to combine scroll methods with wait methods to reliably reveal and interact with scroll-triggered content.
Scroll and Wait for Element to Appear
Combine scrollDown() with waitForElement() to scroll toward
a target element and wait for it to become visible. This pattern handles the common
scenario where an element exists in the DOM but is hidden or transparent until the
user scrolls to its position.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollToRevealElement() {
await session.start();
await session.browser.navigate("https://example.com/landing-page");
await session.browser.wait(2);
await session.browser.scrollDown(1200);
await session.browser.wait(1);
await session.browser.waitForSelector(".pricing-section", 10);
const pricingText = await session.browser.getText(".pricing-section");
console.log("Pricing section content:", pricingText);
await session.close();
}
scrollToRevealElement();
Progressive Scroll to Find Specific Content
When you do not know the exact scroll position of a target element, scroll down incrementally and check for the element after each scroll. This approach works reliably regardless of the page layout and content density because it progressively reveals content until the target is found.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollUntilElementFound(selector, maxScrolls) {
await session.start();
await session.browser.navigate("https://example.com/documentation");
await session.browser.wait(2);
for (let i = 0; i < maxScrolls; i++) {
const found = await session.browser.find(selector);
if (found) {
console.log("Element found after", i + 1, "scrolls");
const elementText = await session.browser.getText(selector);
console.log("Element content:", elementText);
await session.close();
return elementText;
}
await session.browser.scrollDown(400);
await session.browser.wait(0.5);
}
console.log("Element not found after", maxScrolls, "scrolls");
await session.close();
return null;
}
scrollUntilElementFound("#faq-section", 30);
Scroll to Load and Click a Button
Some pages place call-to-action buttons or interactive elements below the fold, requiring the user to scroll down before they can interact with them. This pattern scrolls to the target area, waits for the button to become visible and interactive, then clicks it.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollAndClickButton() {
await session.start();
await session.browser.navigate("https://example.com/special-offer");
await session.browser.wait(2);
await session.browser.scrollDown(1500);
await session.browser.wait(1);
await session.browser.waitForSelector(".claim-offer-btn", 10);
await session.browser.click(".claim-offer-btn");
await session.browser.wait(2);
const resultText = await session.browser.getText(".offer-confirmation");
console.log("Offer result:", resultText);
await session.close();
}
scrollAndClickButton();
Scroll with Data Extraction Pattern
A powerful automation pattern combines scrolling with data extraction to collect information from pages that load content dynamically as the user scrolls. Instead of scrolling through the entire page first and then extracting data, you can extract data incrementally after each scroll, building up your dataset as new content appears. This approach is memory-efficient for very long pages and allows you to process data in real-time as it loads.
Scroll and Extract Product Data
The following example demonstrates scrolling through a product catalog page, extracting product information after each scroll iteration, and building a complete dataset of all products on the page.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function scrollAndExtractProducts() {
await session.start();
await session.browser.navigate("https://example.com/shop/all-products");
await session.browser.wait(3);
let previousLength = 0;
let noChangeCount = 0;
for (let i = 0; i < 30; i++) {
await session.browser.scrollDown(600);
await session.browser.wait(1.5);
const pageText = await session.browser.getText(".product-grid");
const currentLength = pageText.length;
if (currentLength === previousLength) {
noChangeCount++;
if (noChangeCount >= 3) {
console.log("All products loaded after", i + 1, "scrolls");
break;
}
} else {
noChangeCount = 0;
}
previousLength = currentLength;
}
const fullHtml = await session.browser.getHtml(".product-grid");
console.log("Product grid HTML length:", fullHtml.length);
await session.close();
return fullHtml;
}
scrollAndExtractProducts();
Scroll and Collect Social Media Posts
Social media feeds are a classic infinite scroll use case. The following example demonstrates scrolling through a social media feed, collecting post data after each scroll, and stopping when a target number of posts has been collected or the feed has reached its end.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function collectSocialMediaPosts(targetPosts) {
await session.start();
await session.browser.navigate("https://example.com/social/feed");
await session.browser.wait(3);
let collectedCount = 0;
let previousCount = 0;
let staleRounds = 0;
while (collectedCount < targetPosts && staleRounds < 3) {
await session.browser.scrollDown(800);
await session.browser.wait(2);
const feedHtml = await session.browser.getHtml(".feed-container");
collectedCount = (feedHtml.match(/class="post-card"/g) || []).length;
console.log("Posts collected:", collectedCount, "Target:", targetPosts);
if (collectedCount === previousCount) {
staleRounds++;
} else {
staleRounds = 0;
}
previousCount = collectedCount;
}
const finalFeedHtml = await session.browser.getHtml(".feed-container");
console.log("Final feed HTML length:", finalFeedHtml.length);
console.log("Total posts collected:", collectedCount);
await session.close();
return finalFeedHtml;
}
collectSocialMediaPosts(50);
Error Handling
All five scroll methods can throw errors when the session is not in adaptive mode, the session has expired or been closed, or the BBRE engine encounters an internal error while executing the scroll action. The error message contains a description of what went wrong. The following table lists the common errors and their solutions, followed by a comprehensive error handling example.
Error Types
| Error Message | Cause | Solution |
|---|---|---|
BROWSER_ACTION_FAILED |
The scroll action could not be executed by the BBRE engine. This can happen when the browser instance has crashed, the page is unresponsive, or the session is in passive mode. | Verify that the session is running in adaptive mode. Check that the page has finished loading before scrolling. If the browser has crashed, close the session and create a new one. |
SESSION_NOT_FOUND |
The session ID associated with this BBRESession instance does not exist on the server. The session may have been closed or expired. | Check the session status before performing scroll actions. If the session has expired, create a new session and navigate to the target page again. |
SESSION_EXPIRED |
The session has exceeded its maximum lifetime and has been automatically closed by the BBRE engine. | Create a new session with a longer timeout if needed. Implement session renewal logic that creates a new session when the current one expires. |
SESSION_CLOSED |
The session was explicitly closed before the scroll action was attempted. | Ensure that you do not call scroll methods after calling session.close(). Check your code flow to verify that the session is still active. |
Comprehensive Error Handling Example
The following example demonstrates a robust scroll function that handles different error types and implements retry logic for transient failures. This pattern is essential for production automation workflows where reliability is critical.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function reliableScrollDown(pixels, maxRetries) {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
await session.browser.scrollDown(pixels);
return true;
} catch (error) {
const message = error.message.toLowerCase();
if (message.includes("session_expired") || message.includes("session_closed")) {
console.log("Session is no longer active:", error.message);
return false;
}
if (message.includes("session_not_found")) {
console.log("Session not found:", error.message);
return false;
}
if (attempt < maxRetries) {
const delay = 1000 * attempt;
console.log("Scroll attempt", attempt, "failed. Retrying in", delay, "ms");
await new Promise(resolve => setTimeout(resolve, delay));
} else {
console.log("All scroll attempts failed:", error.message);
return false;
}
}
}
return false;
}
async function main() {
await session.start();
await session.browser.navigate("https://example.com/page");
const scrolled = await reliableScrollDown(500, 3);
if (scrolled) {
const content = await session.browser.getText("body");
console.log("Content after scroll:", content.length, "characters");
} else {
console.log("Could not scroll the page");
}
await session.close();
}
main();
Safe Scroll Sequence with Error Recovery
When executing a sequence of multiple scroll actions, wrap the entire sequence in error handling to ensure that a failure in one scroll does not leave the session in an inconsistent state. The following example demonstrates a safe scroll sequence that logs progress and handles errors gracefully.
const BBREClient = require("mydisctsolver-bbre");
const BBRESession = require("mydisctsolver-bbre/session");
const client = new BBREClient({ apiKey: "YOUR_API_KEY" });
const session = new BBRESession(client, { mode: "adaptive" });
async function safeScrollSequence(scrollSteps) {
let completedSteps = 0;
try {
await session.start();
await session.browser.navigate("https://example.com/content");
await session.browser.wait(2);
for (let i = 0; i < scrollSteps; i++) {
try {
await session.browser.scrollDown(500);
await session.browser.wait(1);
completedSteps++;
} catch (scrollError) {
console.log("Scroll failed at step", i + 1, ":", scrollError.message);
break;
}
}
console.log("Completed", completedSteps, "of", scrollSteps, "scroll steps");
const content = await session.browser.getText("body");
return { success: true, completedSteps, contentLength: content.length };
} catch (error) {
console.log("Scroll sequence error:", error.message);
return { success: false, completedSteps, error: error.message };
} finally {
try {
await session.close();
} catch (closeError) {
console.log("Failed to close session:", closeError.message);
}
}
}
safeScrollSequence(15);
Method Comparison
The five scroll methods serve different purposes and are suited to different scenarios. The following table provides a quick comparison to help you choose the right method for your use case.
| Method | Direction | Distance | Best For |
|---|---|---|---|
scroll(options) |
Configurable | Configurable | Custom scroll behavior with engine-specific options. Use when the convenience methods do not cover your needs. |
scrollDown(pixels) |
Down | Relative (default 500px) | Incremental downward scrolling. Ideal for infinite scroll loading, lazy image triggering, and progressive content discovery. |
scrollUp(pixels) |
Up | Relative (default 500px) | Incremental upward scrolling. Useful for returning to previously viewed content and bidirectional scroll patterns. |
scrollToTop() |
Up (absolute) | Absolute (position 0) | Jumping to the page top. Use to reset scroll position, access header elements, or start a new interaction sequence. |
scrollToBottom() |
Down (absolute) | Absolute (page end) | Jumping to the page bottom. Use to access footer content, trigger end-of-page events, or accept terms of service. |
Best Practices
When working with pages that use infinite scroll or progressive content loading,
always use incremental scrollDown() calls with pauses between them
instead of a single scrollToBottom() call. Jumping directly to the
bottom of the page only triggers the scroll event at the final position, which
means intermediate content batches that load based on scroll position thresholds
will be skipped. Incremental scrolling fires scroll events at every position along
the way, ensuring that all content loading triggers are activated. A good starting
point is scrolling 400 to 800 pixels per step with a 1 to 2 second pause between
steps.
After every scroll action, add a wait() call to give the page time
to respond to the scroll event. Websites need time to detect the new scroll
position, fire intersection observer callbacks, initiate network requests for new
content, and render the loaded content into the DOM. Without a pause, your next
action may execute before the scroll-triggered content has loaded, leading to
incomplete data extraction or failed element interactions. The appropriate wait
duration depends on the target website's loading speed: 0.5 to 1 second for fast
sites, 1.5 to 3 seconds for sites that load content via AJAX requests.
Instead of scrolling a fixed number of times and hoping you have loaded all the
content, implement content-change detection by comparing the page content before
and after each scroll. If the content length or item count remains unchanged after
two or three consecutive scroll attempts, the page has likely reached its end.
This approach adapts to pages of any length and avoids both under-scrolling (missing
content) and over-scrolling (wasting time and credits on unnecessary scroll
actions). Track the content length using getText() or count specific
elements using getHtml() with a regex match.
When your automation workflow involves multiple interaction sequences on the same
page (for example, scrolling to load content, then using the search bar, then
scrolling again), call scrollToTop() between sequences to reset the
scroll position to a known state. This makes your scroll calculations predictable
because you always start from position zero. It also ensures that header elements,
navigation menus, and search bars that are positioned at the top of the page are
accessible without guessing how far to scroll up.
When you need to scroll to a specific section of the page and interact with
elements there, combine scroll methods with waitForElement() or
waitForSelector() instead of relying on fixed wait times. After
scrolling, call waitForSelector() with the target element's CSS
selector to wait until the element is actually present and visible in the DOM.
This approach is more reliable than fixed waits because it adapts to varying
page load times and network conditions. If the element does not appear within
the timeout, you know that additional scrolling may be needed.
When implementing scroll loops for infinite scroll pages, always set a maximum iteration limit to prevent your script from scrolling indefinitely. Some pages have truly infinite content (like social media feeds) that will never reach an end. Without a maximum limit, your script will consume credits and time indefinitely. Set a reasonable maximum based on your data requirements, such as 50 scroll iterations or a target item count. Log the scroll progress so you can monitor how many iterations were needed and adjust the limit for future runs.
Common Issues
Symptom: You call scrollToBottom() on an infinite
scroll page, but only the initially loaded content is available. The additional
content batches that should load as you scroll are missing.
Cause: Infinite scroll pages load content in batches triggered
by scroll position thresholds. When you jump directly to the bottom with
scrollToBottom(), the browser fires a single scroll event at the
final position, skipping all intermediate thresholds. The website's JavaScript
only sees the final scroll position and loads one batch (or none) instead of all
the intermediate batches.
Solution: Use incremental scrollDown() calls with
pauses between them instead of scrollToBottom(). Scroll 400 to 800
pixels per step with a 1 to 2 second pause to allow each content batch to load
before triggering the next one.
Symptom: Calling any scroll method throws an error with the
code BROWSER_ACTION_FAILED.
Cause: The session was created in passive mode, which does not
launch a browser instance. Scroll methods require a live browser viewport to
execute scroll actions against, which is only available in adaptive mode.
Solution: Create the session with mode: "adaptive"
to enable browser automation features including scroll methods. If you only need
the raw HTML response without scroll-dependent content, use passive mode with
session.request() instead, which is faster and cheaper.
Symptom: After scrolling through the entire page, some images
still have placeholder src attributes (data URIs or tiny placeholder
URLs) instead of the real image URLs.
Cause: The scroll speed was too fast for the browser to detect
all images entering the viewport and trigger their lazy loading. When you scroll
quickly, images may pass through the viewport detection zone before the
intersection observer has time to fire.
Solution: Reduce the scroll step size to 300 to 400 pixels and
increase the pause duration to 1 to 1.5 seconds between scrolls. This gives the
browser more time to detect each image as it enters the viewport. After
completing the scroll sequence, scroll back to the top and wait an additional 2
to 3 seconds for any remaining images to finish downloading.
Symptom: Your content-change detection logic stops scrolling
because the content length did not change, but you know there is more content
available on the page.
Cause: Some websites have a loading delay between content
batches, or they require the user to scroll past a certain threshold before
triggering the next batch load. If your wait time between scrolls is shorter
than the website's loading time, the content check may run before the new
content has been added to the DOM.
Solution: Increase the wait time between scrolls to 2 to 3
seconds. Also increase the number of consecutive unchanged checks before
stopping from 2 to 4 or 5. This gives the website more time to load content
and reduces false positives in your end-of-content detection.
Symptom: After calling scrollUp() with a specific
pixel value, the page does not appear to be at the expected position. Elements
that should be visible are still out of view.
Cause: The scroll position is relative to the current viewport
position, not absolute. If the page has dynamic content that changes height
(such as collapsible sections, loaded images, or removed elements), the actual
scroll position may differ from what you calculated based on the original page
layout. Additionally, some websites have sticky headers or fixed elements that
consume viewport space, affecting the visible content area.
Solution: Use scrollToTop() to return to a known
absolute position instead of calculating relative scroll distances. If you need
to reach a specific section, use scrollToTop() followed by
scrollDown() with the known distance from the top, or use
find() to locate the target element and scroll incrementally until
it is found.