Using eyeScrollR with fixed areas
Nathanael Larigaldie
2024-02-14
fixed_areas.RmdThe fixed_areas list
Nowadays, many websites have areas that do not move with the rest of the scrolled areas. Typically those are navigation menus at the very top and/or on the right, and footers. The problem with those is that even if the page has been scrolled to the very bottom, gazing at the top menu means that, on the full webpage image, the gaze has suddenly moved from the very bottom of the page to the very top without scrolling back.
For instance, consider this website (which you can find here):
For the moment, let us just ignore the fact that the top area ends up shrinking at some point; we will come back to this later.
The areas at the top and on the right are all fixed, so if you scroll down a bit, only some of the content is actually moving on the screen. The full webpage image may look like this (although the result may slightly differ depending on the plugin you are using to extract the image):
What we really want is to instruct the program that no matter how far down we scrolled, there are areas on the screen that will always be mapped to the same area on the webpage image (while the rest may move around).
In the below example, we show how slightly scrolling down (while gazing at the same content) should only make the program apply a scroll correction to coordinates on the scrollable content (green dot), while redirecting the fixed areas to other fixed areas across the webpage image (red, blue, and yellow dots). As such, fixation on the top right image after a few scrolls should result in the exact same heatmap as on the top left image, before any scroll was made.
The way to achieve this in eyeScrollR is, as the rectangles in the above image suggested, to map a fixed relationship between areas from the screen to areas on the image. Gaze and fixation coordinates in the red rectangle on the screen should always be redirected to the red rectangle on the image, the blue area on the screen to the blue rectangle on the image, and the yellow area to the yellow rectangle on the image.
In a similar way as the manual calibration procedure, just open a screenshot of your screen while browsing the website, and get the coordinates of the top left and bottom right pixels of any fixed area you want to map to your full page image. Once this is done, you can do the same with the destination coordinates on the full page image, as in the following:
Left: the screenshot image; Right: the full webpage image
Repeat the process for every fixed area you may encounter. All that is left to do now is to actually pass these coordinates in a form that the package can recognize. In order to do that, you just need to create pairs of coordinate vectors and pass these to the fixed_areas_bundle function as vectors with the form c(top_left_x, top_left_y, bottom_right_x, bottom_right_y). For example, we measured the following values:
# These two are the top header
top_fixed_area_screen <- c(0, 89, 1919, 276)
top_fixed_area_image <- c(0, 0, 1919, 187)
# These two are the yellowish area on the right
right_fixed_area_screen <- c(1632, 277, 1919, 936)
right_fixed_area_image <- c(1632, 188, 1919, 847)
# These two are the bottom right area
bottom_right_fixed_area_screen <- c(1632, 937, 1919, 1029)
bottom_right_fixed_area_image <- c(1632, 4284, 1919, 4376)
area_bundle <- fixed_areas_bundle(top_fixed_area_screen,
top_fixed_area_image,
right_fixed_area_screen,
right_fixed_area_image,
bottom_right_fixed_area_screen,
bottom_right_fixed_area_image)
fixed_areas <- list(area_bundle)It is imperative to pass the arguments in ordered pairs of screen-to-image mappings, and that you put the result in a list (we will come to the reason why in the next section). The rectangle pairs must also have the exact same dimensions in order to maintain the 1:1 mapping process. If this feels like a long piece of code to write: keep on reading, as a much simplified way to come up with all this code is explained later in this tutorial. All these explanations are here to make sure you understand how the mapping process works.
Once this is done, provided that you already did the calibration procedure, simply pass the fixed_areas argument to the eye_scroll_correct function:
corrected_data <- eye_scroll_correct(eyes_data = test_data,
timestamp_start = 3577,
timestamp_stop = 30864,
image_width = 1920,
image_height = 4377,
calibration = calibration,
fixed_areas = fixed_areas)This should be enough for simple situations with fixed areas that never change.
The rules list
The most complicated use-case scenario that eyeScrollR can handle is when those fixed areas have a somewhat dynamic behavior. The most common example is a top menu bar that shrinks (or even disappears!) when a certain number of pixels have been scrolled down. In these situations, if a screen-to-image direct mapping is still meaningful across the different possible states of the webpage, it is possible to create several bundles of fixed areas mappings and create rules to instruct the package which bundle should be used at what time.
For instance, you could create a bundle of mappings to be used when a user has not scrolled down yet and another one to be used only once they have.
The example we provide is a situation in which the screen-to-image mapping is still meaningful. Indeed, although the top area shrinks, what remains can still be perfectly mapped to the full page image, as the whole area of the shrunken header still has perfect correspondence (as so the bottom right area):
Top: before the shrinking; Bottom: after the shrinking
The right area is a critical-case scenario where the researcher will have to make a decision and may solve things creatively. Indeed, the size of this area once the header has shrunk is now larger than the one we got on the captured full page image. Therefore, we can no longer make a perfect mapping of the two areas. In some cases, this might mean that it is impossible to map everything in a single image, and you might consider splitting your participant’s data to have several images.
However, in this particular case, we can choose to ignore this, or even do some manual editing to the full page image to expand the area! Indeed, the empty area below the one we are interested in is otherwise never reachable to an observer: there will always be something on the right of the page, and the long band of white is only an artifact of the screenshot plugin. As such, it is possible to just make the following mapping:
In this situation, the bottom-right coordinates of the fixed area on the full page image can be calculated by making sure that the two areas (the one on the screen and the one on the image) have the exact same size.
In our case, we ended up with the following measures:
# These two are the top header
top_fixed_area_bis_screen <- c(0, 89, 1919, 182)
top_fixed_area_bis_image <- c(0, 0, 1919, 93)
# These two are the yellowish area on the right
right_fixed_area_bis_screen <- c(1632, 183, 1919, 936)
right_fixed_area_bis_image <- c(1632, 188, 1919, 941)
# These two are the bottom right area
bottom_right_fixed_area_bis_screen <- c(1632, 937, 1919, 1029)
bottom_right_fixed_area_bis_image <- c(1632, 4284, 1919, 4376)
area_bundle_bis <- fixed_areas_bundle(top_fixed_area_bis_screen,
top_fixed_area_bis_image,
right_fixed_area_bis_screen,
right_fixed_area_bis_image,
bottom_right_fixed_area_bis_screen,
bottom_right_fixed_area_bis_image)
fixed_areas <- list(area_bundle, area_bundle_bis)This time, the fixed_areas list comprises two bundles of mappings: the one that should be used before the user has scrolled down enough to shrink the top header, and the one that should be used after.
The only thing left to do is to create a rules list, so that the package can know when to use which bundle.
A rule is a function that checks, at each line of the .csv file, if conditions to use a specific bundle are met or not. In our case here, we would need a first rule asking if the user has not scrolled down a sufficient amount of pixels to shrink the header, in which case the first bundle of fixed areas should be used, and not otherwise. We would also require a second rule, asking the exact opposite for the second bundle. Finding the conditions that change the fixed areas may require some experimentation in the field. In our website example, the header just retracts when more than 1000 pixels have been scrolled down, and expands again in the other case.
The eye_scroll_correct function calls all rule functions at every line of the .csv file and pass them several arguments to work on. Working on complex sets of rules is possible if you have the programming knowledge to do so. In any case, it will require you to write your own functions. The scenario presented here is easy enough to be modified whatever your level of expertise in R is and should be appropriate for the vast majority of the use-case scenarios.
rule_scrolled_up <- function (data_line, array_fixed_areas, flag, scroll)
{
if (scroll <= 1000)
{
return (TRUE)
}
else
{
return (FALSE)
}
}
rule_scrolled_down <- function (data_line, array_fixed_areas, flag, scroll)
{
if (scroll > 1000)
{
return (TRUE)
}
else
{
return (FALSE)
}
}
rules <- list(rule_scrolled_up, rule_scrolled_down)The order of these rules must be the same as the order of the fixed areas bundles. In our case, the rule_scrolled_up rule will therefore make sure that the first bundle of fixed areas will be used when less than 1000 pixels have been scrolled down and no longer be used should it not be the case. Similarly, the rule_scrolled_down rule will be associated with the second bundle of fixed areas. You can copy and paste these functions and simply change the number 1000 in each of them to fit your needs if you are in a similar case.
If this all feels a bit overwhelming, don’t worry: we’ve got you covered. Instead of writing everything by hand, you can just use the bundle creator gadget by running the following code:
This gadget simplifies the process by letting you focus on what you are doing instead of how. You can add/remove fixed areas bundles with a simple click, fill in the fixed areas coordinates, and select which (common) rule you need for each bundle:
This will output some code that you can copy & paste, and the only thing left you have to do is modify the rules to suit your needs. Just don’t forget to click “done” once you are finished!
You can then just pass the list of fixed areas and the list of rules to the eye_scroll_correct function:
corrected_data <- eye_scroll_correct(eyes_data = test_data,
timestamp_start = 3577,
timestamp_stop = 30864,
image_width = 1920,
image_height = 4377,
calibration = calibration,
fixed_areas = fixed_areas,
rules = rules,
scroll_lag = (1/120)*1000)A complete, commented code for our example
library(eyeScrollR)
library(png)
library(tidyverse)
################################################################################
# General settings
# Same for every data file for a given webpage
# (Do this once per webpage)
################################################################################
### Automatic calibration
calibration_image <- readPNG("calibration_chrome.png")
calibration <- scroll_calibration_auto(calibration_image, 125)
### Fixed area bundles
## First bundle: when not scrolled down yet
# These two are the top header
top_fixed_area_screen <- c(0, 89, 1919, 276)
top_fixed_area_image <- c(0, 0, 1919, 187)
# These two are the yellowish area on the right
right_fixed_area_screen <- c(1632, 277, 1919, 936)
right_fixed_area_image <- c(1632, 188, 1919, 847)
# These two are the bottom right area
bottom_right_fixed_area_screen <- c(1632, 937, 1919, 1029)
bottom_right_fixed_area_image <- c(1632, 4284, 1919, 4376)
area_bundle <- fixed_areas_bundle(top_fixed_area_screen,
top_fixed_area_image,
right_fixed_area_screen,
right_fixed_area_image,
bottom_right_fixed_area_screen,
bottom_right_fixed_area_image)
## Second bundle: when scrolled down enough
# These two are the top header
top_fixed_area_bis_screen <- c(0, 89, 1919, 182)
top_fixed_area_bis_image <- c(0, 0, 1919, 93)
# These two are the yellowish area on the right
right_fixed_area_bis_screen <- c(1632, 183, 1919, 936)
right_fixed_area_bis_image <- c(1632, 188, 1919, 941)
# These two are the bottom right area
bottom_right_fixed_area_bis_screen <- c(1632, 937, 1919, 1029)
bottom_right_fixed_area_bis_image <- c(1632, 4284, 1919, 4376)
area_bundle_bis <- fixed_areas_bundle(top_fixed_area_bis_screen,
top_fixed_area_bis_image,
right_fixed_area_bis_screen,
right_fixed_area_bis_image,
bottom_right_fixed_area_bis_screen,
bottom_right_fixed_area_bis_image)
### Now, the rule functions
# The first one, when we are below 1000 pixels scrolled
rule_scrolled_up <- function (data_line, array_fixed_areas, flag, scroll)
{
if (scroll <= 1000)
{
return (TRUE)
}
else
{
return (FALSE)
}
}
# The second one, when we are above 1000 pixels scrolled
rule_scrolled_down <- function (data_line, array_fixed_areas, flag, scroll)
{
if (scroll > 1000)
{
return (TRUE)
}
else
{
return (FALSE)
}
}
# Create the fixed_areas list with both bundles, and associate rules by putting
# them in the same order
fixed_areas <- list(area_bundle, area_bundle_bis)
rules <- list(rule_scrolled_up, rule_scrolled_down)
################################################################################
# Participant-specific code
################################################################################
# Read data, and make sure labels are alright
test_data <- read_csv("fixed_areas.csv")
# Apply the scroll corrections
corrected_data <- eye_scroll_correct(eyes_data = test_data,
timestamp_start = 8504,
timestamp_stop = 70478,
image_width = 1920,
image_height = 4377,
calibration = calibration,
fixed_areas = fixed_areas,
rules = rules,
scroll_lag = (1/120)*1000)
# Read the full page image
heatmap_image <- readPNG("fixed_website.png")
# Output the heatmap
generate_heatmap(data = corrected_data, heatmap_image = heatmap_image)