Be notified in real-time on any activity (for instance connection, cursor move, typing...) #3

Open
opened 2024-03-01 17:34:56 +01:00 by Benjamin_Loison · 43 comments

This would enable us to backup Overleaf content without regular git pull, see OC3K crontab.

Related to #11, Overleaf_history_exporter and Improve_websites_thanks_to_open_source/issues/770.

+72

This would enable us to backup Overleaf content without regular `git pull`, see OC3K `crontab`. Related to #11, [Overleaf_history_exporter](https://codeberg.org/Benjamin_Loison/Overleaf_history_exporter) and [Improve_websites_thanks_to_open_source/issues/770](https://codeberg.org/Benjamin_Loison/Improve_websites_thanks_to_open_source/issues/770). +72
Benjamin_Loison changed title from Be notified in real-time on any activity (for instance cursor move, typing...) to Be notified in real-time on any activity (for instance connection, cursor move, typing...) 2024-03-01 20:44:07 +01:00
Author
Owner

DuckDuckGo and Google search Overleaf WebSocket.

DuckDuckGo and Google search *Overleaf WebSocket*.
Author
Owner

Most results seem about hosting our own instance.

https://www.overleaf.com/socket.io/1/websocket/

XXXXXXXXXXXXXXXXXXXX:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling

first part changes everytime. Same for https://www.overleaf.com/socket.io/1/ and https://www.overleaf.com/socket.io/1/uzgxahft/ uzgxahft being a random string I generated.

There is no further request on Firefox when loading this webpage.

Most results seem about hosting our own instance. https://www.overleaf.com/socket.io/1/websocket/ ``` XXXXXXXXXXXXXXXXXXXX:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling ``` first part changes everytime. Same for https://www.overleaf.com/socket.io/1/ and https://www.overleaf.com/socket.io/1/uzgxahft/ `uzgxahft` being a random string I generated. There is no further request on Firefox when loading this webpage.
Author
Owner

Should investigate these keywords. Well these seem to be old technologies except websocket.

Should investigate these keywords. Well these seem to be old technologies except `websocket`.
Author
Owner

Let us read Wikipedia: WebSocket (1265617417). I quite completely read it.

Let us read [Wikipedia: WebSocket (1265617417)](https://en.wikipedia.org/w/index.php?title=WebSocket&oldid=1265617417). I quite completely read it.
Author
Owner
Related to [Benjamin-Loison/Contracts-management-website/issues/1](https://github.com/Benjamin-Loison/Contracts-management-website/issues/1).
Author
Owner
curl -I https://www.overleaf.com/socket.io/1/websocket/
Output:
HTTP/2 200 
content-type: text/plain
date: Sun, 02 Mar 2025 23:30:05 GMT
via: 1.1 google
set-cookie: GCLB=XXXXXXXXXXXXXXXX; path=/; HttpOnly
alt-svc: clear

```bash curl -I https://www.overleaf.com/socket.io/1/websocket/ ``` <details> <summary>Output:</summary> ``` HTTP/2 200 content-type: text/plain date: Sun, 02 Mar 2025 23:30:05 GMT via: 1.1 google set-cookie: GCLB=XXXXXXXXXXXXXXXX; path=/; HttpOnly alt-svc: clear ``` </details>
Author
Owner

DuckDuckGo search "Protocol "wss" not supported or disabled in libcurl".

The Stack Overflow answer 47861907:

Shell:
benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c "wss://ws-feed.gdax.com"
error: Unexpected server response: 526
> benjamin_loison@Benjamin-Loison-HP-Debian:~$
Shell:
benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/'
error: Unexpected server response: 502
> benjamin_loison@Benjamin-Loison-HP-Debian:~$
Shell:
benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/'
Connected (press CTRL+C to quit)
< 7:::1+0
Disconnected (code: 1006, reason: "")
DuckDuckGo search `"Protocol "wss" not supported or disabled in libcurl"`. [The Stack Overflow answer 47861907](https://stackoverflow.com/a/47861907): <details> <summary>Shell:</summary> ``` benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c "wss://ws-feed.gdax.com" error: Unexpected server response: 526 > benjamin_loison@Benjamin-Loison-HP-Debian:~$ ``` </details> <details> <summary>Shell:</summary> ``` benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/' error: Unexpected server response: 502 > benjamin_loison@Benjamin-Loison-HP-Debian:~$ ``` </details> <details> <summary>Shell:</summary> ``` benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/' Connected (press CTRL+C to quit) < 7:::1+0 Disconnected (code: 1006, reason: "") ``` </details>
Author
Owner
Shell:
benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX'
error: Unexpected server response: 502
> benjamin_loison@Benjamin-Loison-HP-Debian:~$
Shell:
benjamin_loison@Benjamin-Loison-HP-Debian:~$  wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX'
Connected (press CTRL+C to quit)
Disconnected (code: 1006, reason: "")
<details> <summary>Shell:</summary> ``` benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX' error: Unexpected server response: 502 > benjamin_loison@Benjamin-Loison-HP-Debian:~$ ``` </details> <details> <summary>Shell:</summary> ``` benjamin_loison@Benjamin-Loison-HP-Debian:~$ wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX' Connected (press CTRL+C to quit) Disconnected (code: 1006, reason: "") ``` </details>
Author
Owner
Bash script:
 curl 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=YYYYYYYYYYYYYYYYYYYYYYYY' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br, zstd' -H 'Sec-WebSocket-Version: 13' -H 'Origin: https://www.overleaf.com' -H 'Sec-WebSocket-Extensions: permessage-deflate' -H 'Sec-WebSocket-Key: ZZZZZZZZZZZZZZZZZZZZZZZZ' -H 'Connection: keep-alive, Upgrade' -H 'Cookie: oa=0; overleaf_session2=s%3AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB%2BCCCCCC; GCLB=DDDDDDDDDDDDDDDD' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: websocket' -H 'Sec-Fetch-Site: same-origin' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade: websocket'
curl: (1) Protocol "wss" not supported or disabled in libcurl
<details> <summary>Bash script:</summary> ```bash curl 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=YYYYYYYYYYYYYYYYYYYYYYYY' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br, zstd' -H 'Sec-WebSocket-Version: 13' -H 'Origin: https://www.overleaf.com' -H 'Sec-WebSocket-Extensions: permessage-deflate' -H 'Sec-WebSocket-Key: ZZZZZZZZZZZZZZZZZZZZZZZZ' -H 'Connection: keep-alive, Upgrade' -H 'Cookie: oa=0; overleaf_session2=s%3AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB%2BCCCCCC; GCLB=DDDDDDDDDDDDDDDD' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: websocket' -H 'Sec-Fetch-Site: same-origin' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade: websocket' ``` </details> ``` curl: (1) Protocol "wss" not supported or disabled in libcurl ```
Author
Owner

https://socket.io

socket.io

About

Realtime application framework (Node.JS server)

so it won't help much it seems.

https://socket.io/docs/v4/client-api/ may help.

https://socket.io [socket.io](https://github.com/socketio/socket.io) > About > Realtime application framework (Node.JS server) so it won't help much it seems. https://socket.io/docs/v4/client-api/ may help.
Author
Owner
 wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX'
Output:
Connected (press CTRL+C to quit)
< 7:::1+0
Disconnected (code: 1006, reason: "")
```bash wscat -c 'wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX' ``` <details> <summary>Output:</summary> ``` Connected (press CTRL+C to quit) < 7:::1+0 Disconnected (code: 1006, reason: "") ``` </details>
Author
Owner

DuckDuckGo search socket.io Python client.

DuckDuckGo search *socket.io Python client*.
Author
Owner

https://pypi.org/project/python-socketio/
python-socketio has 4.1 k stars and last commit was 3 days ago.

https://pypi.org/project/python-socketio/ [python-socketio](https://github.com/miguelgrinberg/python-socketio) has 4.1 k stars and last commit was 3 days ago.
Author
Owner

The Socket.IO protocol has been through a number of revisions, and some of these introduced backward incompatible changes, which means that the client and the server must use compatible versions for everything to work.

Source: python-socketio/blob/781dc9a0305f6a795a0467ebc795f4b765084a0d/README.md#version-compatibility

> The Socket.IO protocol has been through a number of revisions, and some of these introduced backward incompatible changes, which means that the client and the server must use compatible versions for everything to work. Source: [python-socketio/blob/781dc9a0305f6a795a0467ebc795f4b765084a0d/README.md#version-compatibility](https://github.com/miguelgrinberg/python-socketio/blob/781dc9a0305f6a795a0467ebc795f4b765084a0d/README.md#version-compatibility)
Author
Owner
https://python-socketio.readthedocs.io/en/stable/client.html
Author
Owner
pip install "python-socketio[client]"
Output:
Requirement already satisfied: python-socketio[client] in ./venv/lib/python3.11/site-packages (5.11.3)
Requirement already satisfied: bidict>=0.21.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (0.23.1)
Requirement already satisfied: python-engineio>=4.8.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (4.11.2)
Requirement already satisfied: requests>=2.21.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (2.32.3)
Requirement already satisfied: websocket-client>=0.54.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (1.8.0)
Requirement already satisfied: simple-websocket>=0.10.0 in ./venv/lib/python3.11/site-packages (from python-engineio>=4.8.0->python-socketio[client]) (1.1.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (2025.1.31)
Requirement already satisfied: wsproto in ./venv/lib/python3.11/site-packages (from simple-websocket>=0.10.0->python-engineio>=4.8.0->python-socketio[client]) (1.2.0)
Requirement already satisfied: h11<1,>=0.9.0 in ./venv/lib/python3.11/site-packages (from wsproto->simple-websocket>=0.10.0->python-engineio>=4.8.0->python-socketio[client]) (0.14.0)
```bash pip install "python-socketio[client]" ``` <details> <summary>Output:</summary> ``` Requirement already satisfied: python-socketio[client] in ./venv/lib/python3.11/site-packages (5.11.3) Requirement already satisfied: bidict>=0.21.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (0.23.1) Requirement already satisfied: python-engineio>=4.8.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (4.11.2) Requirement already satisfied: requests>=2.21.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (2.32.3) Requirement already satisfied: websocket-client>=0.54.0 in ./venv/lib/python3.11/site-packages (from python-socketio[client]) (1.8.0) Requirement already satisfied: simple-websocket>=0.10.0 in ./venv/lib/python3.11/site-packages (from python-engineio>=4.8.0->python-socketio[client]) (1.1.0) Requirement already satisfied: charset-normalizer<4,>=2 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (2.3.0) Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.11/site-packages (from requests>=2.21.0->python-socketio[client]) (2025.1.31) Requirement already satisfied: wsproto in ./venv/lib/python3.11/site-packages (from simple-websocket>=0.10.0->python-engineio>=4.8.0->python-socketio[client]) (1.2.0) Requirement already satisfied: h11<1,>=0.9.0 in ./venv/lib/python3.11/site-packages (from wsproto->simple-websocket>=0.10.0->python-engineio>=4.8.0->python-socketio[client]) (0.14.0) ``` </details>
Author
Owner
Python script:
import socketio

# standard Python
with socketio.SimpleClient() as sio:
    # ... connect to a server and use the client
    # ... no need to manually disconnect!
    sio.connect('https://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX', transports = ['websocket'])
    event = sio.receive()
    print(f'received event: "{event[0]}" with arguments {event[1:]}')
Output:
Traceback (most recent call last):
  File "<tmp 3>", line 7, in <module>
    sio.connect('wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX', transports = ['websocket'])
  File "/home/benjamin_loison/venv/lib/python3.11/site-packages/socketio/simple_client.py", line 82, in connect
    self.client.connect(url, headers=headers, auth=auth,
  File "/home/benjamin_loison/venv/lib/python3.11/site-packages/socketio/client.py", line 159, in connect
    raise exceptions.ConnectionError(exc.args[0]) from None
socketio.exceptions.ConnectionError: Connection error

same if use https://.

<details> <summary>Python script:</summary> ```python import socketio # standard Python with socketio.SimpleClient() as sio: # ... connect to a server and use the client # ... no need to manually disconnect! sio.connect('https://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX', transports = ['websocket']) event = sio.receive() print(f'received event: "{event[0]}" with arguments {event[1:]}') ``` </details> <details> <summary>Output:</summary> ``` Traceback (most recent call last): File "<tmp 3>", line 7, in <module> sio.connect('wss://www.overleaf.com/socket.io/1/websocket/XXXXXXXXXXXXXXXXXXXX?projectId=XXXXXXXXXXXXXXXXXXXXXXXX', transports = ['websocket']) File "/home/benjamin_loison/venv/lib/python3.11/site-packages/socketio/simple_client.py", line 82, in connect self.client.connect(url, headers=headers, auth=auth, File "/home/benjamin_loison/venv/lib/python3.11/site-packages/socketio/client.py", line 159, in connect raise exceptions.ConnectionError(exc.args[0]) from None socketio.exceptions.ConnectionError: Connection error ``` </details> same if use `https://`.
Author
Owner

DuckDuckGo and Google search Python Overleaf client.

pyoverleaf does not seem helpful.

DuckDuckGo and Google search *Python Overleaf client*. [pyoverleaf](https://github.com/jkulhanek/pyoverleaf) does not seem helpful.
Author
Owner

https://www.overleaf.com/socket.io/

Welcome to socket.io.

https://www.overleaf.com/socket.io/ *Welcome to socket.io.*
Author
Owner

https://plmlatex.math.cnrs.fr/project let see when and who did last edit.

https://plmlatex.math.cnrs.fr/project let see when and who did last edit.
Author
Owner

May help #21.

May help #21.
Author
Owner

The Stack Overflow answer 4152986 helps to add some logs.

[The Stack Overflow answer 4152986](https://stackoverflow.com/a/4152986) helps to add some logs.
Author
Owner
notifyOnNewOverleafActivity.py.log:
2025-09-30 15:35:50.061342 ...
...
2025-10-02 07:59:30.409043 message='2::'
2025-10-02 07:59:56.346952 message='2::'
2025-10-02 08:03:14.232271 message='1::'
<details> <summary><code>notifyOnNewOverleafActivity.py.log</code>:</summary> ``` 2025-09-30 15:35:50.061342 ... ... 2025-10-02 07:59:30.409043 message='2::' 2025-10-02 07:59:56.346952 message='2::' 2025-10-02 08:03:14.232271 message='1::' ``` </details>
Author
Owner
'5:::{"name":"clientTracking.clientUpdated","args":[{"row":14,"column":12,"doc_id":"XXXXXXXXXXXXXXXXXXXXXXXX","id":"X.XXXXXXXXXXXXXXXXXXXX","user_id":"XXXXXXXXXXXXXXXXXXXXXXXX","email":"benjamin.loison@CENSORED.fr","name":"Benjamin Loison"}]}'
``` '5:::{"name":"clientTracking.clientUpdated","args":[{"row":14,"column":12,"doc_id":"XXXXXXXXXXXXXXXXXXXXXXXX","id":"X.XXXXXXXXXXXXXXXXXXXX","user_id":"XXXXXXXXXXXXXXXXXXXXXXXX","email":"benjamin.loison@CENSORED.fr","name":"Benjamin Loison"}]}' ```
Author
Owner
head -c 27 notifyOnNewRealTimeOverleafActivity.py.log 
2025-09-30 15:35:50.061342 
grep -v "message='2::'" notifyOnNewRealTimeOverleafActivity.py.log | tail -n 1 | head -c 27
2025-10-06 14:53:09.098792 
tail -n 3 notifyOnNewRealTimeOverleafActivity.py.log
Output:
2025-10-06 15:01:00.706347 message='2::'
2025-10-06 15:01:25.734648 message='2::'
2025-10-06 15:01:50.762080 message='2::'

So it seems to have broke after 8 minutes and 41 seconds.

Note that in theory there is no need to do anything as I just read the network of a relatively regular web-browser.

``` head -c 27 notifyOnNewRealTimeOverleafActivity.py.log ``` ``` 2025-09-30 15:35:50.061342 ``` ``` grep -v "message='2::'" notifyOnNewRealTimeOverleafActivity.py.log | tail -n 1 | head -c 27 ``` ``` 2025-10-06 14:53:09.098792 ``` ``` tail -n 3 notifyOnNewRealTimeOverleafActivity.py.log ``` <details> <summary>Output:</summary> ``` 2025-10-06 15:01:00.706347 message='2::' 2025-10-06 15:01:25.734648 message='2::' 2025-10-06 15:01:50.762080 message='2::' ``` </details> So it seems to have broke after 8 minutes and 41 seconds. Note that in theory there is no need to do anything as I just read the network of a relatively regular web-browser.
Author
Owner

Check if no response since 50 seconds seems to be a good way to be notified when the WebSocket does not work anymore.

Alternatively should check if after the same amount of time face again this issue.

Check if no response since 50 seconds seems to be a good way to be notified when the WebSocket does not work anymore. Alternatively should check if after the same amount of time face again this issue.
Author
Owner
ps aux | grep '[n]otifyOnNewRealTimeOverleafActivity.py'
Output:
benjami+    1648  0.0  0.0   7764  3016 ?        Ss   Oct05   0:00 /usr/bin/bash /usr/bin/my_cron_shell -c cd notifications/ && ~/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py
benjami+    1676  0.0  0.0   7024   880 ?        S    Oct05   0:00 /usr/bin/bash /usr/bin/my_cron_shell -c cd notifications/ && ~/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py
benjami+    1679  0.0  0.3  43384 29600 ?        S    Oct05   0:11 /home/benjamin_loison/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py
pkill -f notifyOnNewRealTimeOverleafActivity.py
ps aux | grep '[n]otifyOnNewRealTimeOverleafActivity.py'

Would be interesting to see the window to see if there is any prompt.

Note that my approach may not be stable if the server is not stable, as a regular user may see a prompt, possibly with a button, or just reload the webpage.

```bash ps aux | grep '[n]otifyOnNewRealTimeOverleafActivity.py' ``` <details> <summary>Output:</summary> ``` benjami+ 1648 0.0 0.0 7764 3016 ? Ss Oct05 0:00 /usr/bin/bash /usr/bin/my_cron_shell -c cd notifications/ && ~/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py benjami+ 1676 0.0 0.0 7024 880 ? S Oct05 0:00 /usr/bin/bash /usr/bin/my_cron_shell -c cd notifications/ && ~/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py benjami+ 1679 0.0 0.3 43384 29600 ? S Oct05 0:11 /home/benjamin_loison/venv/bin/python3 notifyOnNewRealTimeOverleafActivity.py ``` </details> ``` pkill -f notifyOnNewRealTimeOverleafActivity.py ``` ```bash ps aux | grep '[n]otifyOnNewRealTimeOverleafActivity.py' ``` Would be interesting to see the window to see if there is any prompt. Note that my approach may not be stable if the server is not stable, as a regular user may see a prompt, possibly with a button, or just reload the webpage.
Author
Owner
Python script:
import time
import threading

UPDATE_DELAY = 1
THRESHOLD = UPDATE_DELAY * 2
NORMAL_BEHAVIOR_PERIOD = 3 * THRESHOLD
lastTime = None

def verifyLastTime():
    while True:
        currentTime = time.time()
        print(f'Checking at {currentTime}')
        if lastTime is not None and lastTime < currentTime - THRESHOLD:
            print('No more update!')
            break
        # Sleeping just enough depending on `lastTime` may not be efficient, but mid range would be interesting.
        time.sleep(UPDATE_DELAY)

thread = threading.Thread(target = verifyLastTime)
thread.start()

for _ in range(NORMAL_BEHAVIOR_PERIOD):
    lastTime = time.time()
    print(f'{lastTime=}')
    time.sleep(UPDATE_DELAY)

thread.join()
Output:
Checking at 1759915873.3947287
lastTime=1759915873.3947918
lastTime=1759915874.3950238
Checking at 1759915874.3957164
lastTime=1759915875.3955274
Checking at 1759915875.3961875
lastTime=1759915876.3960567
Checking at 1759915876.3965623
lastTime=1759915877.396563
Checking at 1759915877.396842
lastTime=1759915878.3968308
Checking at 1759915878.3970864
Checking at 1759915879.3973584
Checking at 1759915880.3978689
No more update!

works as wanted.

<details> <summary>Python script:</summary> ```python import time import threading UPDATE_DELAY = 1 THRESHOLD = UPDATE_DELAY * 2 NORMAL_BEHAVIOR_PERIOD = 3 * THRESHOLD lastTime = None def verifyLastTime(): while True: currentTime = time.time() print(f'Checking at {currentTime}') if lastTime is not None and lastTime < currentTime - THRESHOLD: print('No more update!') break # Sleeping just enough depending on `lastTime` may not be efficient, but mid range would be interesting. time.sleep(UPDATE_DELAY) thread = threading.Thread(target = verifyLastTime) thread.start() for _ in range(NORMAL_BEHAVIOR_PERIOD): lastTime = time.time() print(f'{lastTime=}') time.sleep(UPDATE_DELAY) thread.join() ``` </details> <details> <summary>Output:</summary> ``` Checking at 1759915873.3947287 lastTime=1759915873.3947918 lastTime=1759915874.3950238 Checking at 1759915874.3957164 lastTime=1759915875.3955274 Checking at 1759915875.3961875 lastTime=1759915876.3960567 Checking at 1759915876.3965623 lastTime=1759915877.396563 Checking at 1759915877.396842 lastTime=1759915878.3968308 Checking at 1759915878.3970864 Checking at 1759915879.3973584 Checking at 1759915880.3978689 No more update! ``` </details> works as wanted.
Author
Owner
[Benjamin_Loison/playwright/issues/3](https://codeberg.org/Benjamin_Loison/playwright/issues/3) would help.
Author
Owner
Output:
...
2025-10-09 00:40:37.703382 Checking at 1759963237.7033584
2025-10-09 00:40:40.789355 message='2::'
2025-10-09 00:41:02.703878 Checking at 1759963262.7038379
2025-10-09 00:41:05.821398 message='2::'
2025-10-09 00:41:27.704372 Checking at 1759963287.704344
2025-10-09 00:41:30.851719 message='2::'
<details> <summary>Output:</summary> ``` ... 2025-10-09 00:40:37.703382 Checking at 1759963237.7033584 2025-10-09 00:40:40.789355 message='2::' 2025-10-09 00:41:02.703878 Checking at 1759963262.7038379 2025-10-09 00:41:05.821398 message='2::' 2025-10-09 00:41:27.704372 Checking at 1759963287.704344 2025-10-09 00:41:30.851719 message='2::' ``` </details>
Author
Owner
grep '2::' notifyOnNewRealTimeOverleafActivity.py.log | tail -n 3
Output:
2025-10-10 00:41:14.054577 message='2::'
2025-10-10 00:41:39.084075 message='2::'
2025-10-10 00:42:04.114034 message='2::'
```bash grep '2::' notifyOnNewRealTimeOverleafActivity.py.log | tail -n 3 ``` <details> <summary>Output:</summary> ``` 2025-10-10 00:41:14.054577 message='2::' 2025-10-10 00:41:39.084075 message='2::' 2025-10-10 00:42:04.114034 message='2::' ``` </details>
Author
Owner
tail -n 3 notifyOnNewRealTimeOverleafActivity.py.log
Output:
2025-10-13 16:15:02.594978 Checking at 1760364902.5949585
2025-10-13 16:15:27.595367 Checking at 1760364927.5953403
2025-10-13 16:15:52.595749 Checking at 1760364952.5957308
grep 'No more Overleaf ping' notifyOnNewRealTimeOverleafActivity.py.log

does not return anything.

An alternative would to schedule a notification function and abort it and start a new one when receive a heartbeat.

grep 'message' notifyOnNewRealTimeOverleafActivity.py.log | tail -n 3
Output:
2025-10-10 00:41:14.054577 message='2::'
2025-10-10 00:41:39.084075 message='2::'
2025-10-10 00:42:04.114034 message='2::'

Now using:

print(f'Checking at {currentTime} ({lastTime=})')
./notifyOnNewRealTimeOverleafActivity.py
Output:
2025-10-13 16:19:53.445716 message='6:::3+[null,[{"last_updated_at":"1760365192431","user_id":"XXXXXXXXXXXXXXXXXXXXXXXX","first_name":"Benjamin","last_name":"Loison","email":"benjamin.loison@CENSORED.fr","cursorData":{"row":1088,"column":0,"doc_id":"XXXXXXXXXXXXXXXXXXXXXXXX"},"connected":true,"client_id":"P.XX_XXXXXXXXXXXXXXXXX","client_age":1.001},{"last_updated_at":"1760365192431","user_id":"anonymous-user","first_name":"","last_name":"","email":"","connected":true,"client_id":"P.XXXXXXXXXXXXXXXXXX-X","client_age":1.001}]]'
2025-10-13 16:19:58.916655 Checking at 1760365198.9166238 (lastTime=None)
2025-10-13 16:20:02.831867 message='6:::2+[{"message":"Something went wrong in real-time service"}]'
2025-10-13 16:20:03.585845 message='2::'
2025-10-13 16:20:23.917074 Checking at 1760365223.9170516 (lastTime=None)
2025-10-13 16:20:28.617499 message='2::'
2025-10-13 16:20:48.917365 Checking at 1760365248.9173453 (lastTime=None)
``` tail -n 3 notifyOnNewRealTimeOverleafActivity.py.log ``` <details> <summary>Output:</summary> ``` 2025-10-13 16:15:02.594978 Checking at 1760364902.5949585 2025-10-13 16:15:27.595367 Checking at 1760364927.5953403 2025-10-13 16:15:52.595749 Checking at 1760364952.5957308 ``` </details> ```bash grep 'No more Overleaf ping' notifyOnNewRealTimeOverleafActivity.py.log ``` does not return anything. An alternative would to schedule a notification function and abort it and start a new one when receive a heartbeat. ```bash grep 'message' notifyOnNewRealTimeOverleafActivity.py.log | tail -n 3 ``` <details> <summary>Output:</summary> ``` 2025-10-10 00:41:14.054577 message='2::' 2025-10-10 00:41:39.084075 message='2::' 2025-10-10 00:42:04.114034 message='2::' ``` </details> Now using: ```python print(f'Checking at {currentTime} ({lastTime=})') ``` ```bash ./notifyOnNewRealTimeOverleafActivity.py ``` <details> <summary>Output:</summary> ``` 2025-10-13 16:19:53.445716 message='6:::3+[null,[{"last_updated_at":"1760365192431","user_id":"XXXXXXXXXXXXXXXXXXXXXXXX","first_name":"Benjamin","last_name":"Loison","email":"benjamin.loison@CENSORED.fr","cursorData":{"row":1088,"column":0,"doc_id":"XXXXXXXXXXXXXXXXXXXXXXXX"},"connected":true,"client_id":"P.XX_XXXXXXXXXXXXXXXXX","client_age":1.001},{"last_updated_at":"1760365192431","user_id":"anonymous-user","first_name":"","last_name":"","email":"","connected":true,"client_id":"P.XXXXXXXXXXXXXXXXXX-X","client_age":1.001}]]' 2025-10-13 16:19:58.916655 Checking at 1760365198.9166238 (lastTime=None) 2025-10-13 16:20:02.831867 message='6:::2+[{"message":"Something went wrong in real-time service"}]' 2025-10-13 16:20:03.585845 message='2::' 2025-10-13 16:20:23.917074 Checking at 1760365223.9170516 (lastTime=None) 2025-10-13 16:20:28.617499 message='2::' 2025-10-13 16:20:48.917365 Checking at 1760365248.9173453 (lastTime=None) ``` </details>
Author
Owner

global lastTime was missing in server_message_handler.

`global lastTime` was missing in `server_message_handler`.
Author
Owner
grep -vE "(Checking at |message='2::')" notifyOnNewRealTimeOverleafActivity.py.log | tail -n 1
2025-10-18 18:12:23.204476 message='6:::3+[null,[{"last_updated_at":"1760803942124","user_id":"anonymous-user","first_name":"","last_name":"","email":"","connected":true,"client_id":"P.XXXXXXXXXXXXXXXXXXXX","client_age":1.003}]]'
tail -n 6 notifyOnNewRealTimeOverleafActivity.py.log
Output:
2025-10-19 18:11:32.605908 message='2::'
2025-10-19 18:11:50.161417 Checking at 1760890310.1613894 (lastTime=1760890292.6060472)
2025-10-19 18:11:57.633351 message='2::'
2025-10-19 18:12:15.161793 Checking at 1760890335.161772 (lastTime=1760890317.6334727)
2025-10-19 18:12:40.162049 Checking at 1760890360.162032 (lastTime=1760890317.6334727)
2025-10-19 18:13:05.162317 Checking at 1760890385.1622937 (lastTime=1760890317.6334727)

It seems correct as I restarted yesterday OverClock3000.

```bash grep -vE "(Checking at |message='2::')" notifyOnNewRealTimeOverleafActivity.py.log | tail -n 1 ``` ``` 2025-10-18 18:12:23.204476 message='6:::3+[null,[{"last_updated_at":"1760803942124","user_id":"anonymous-user","first_name":"","last_name":"","email":"","connected":true,"client_id":"P.XXXXXXXXXXXXXXXXXXXX","client_age":1.003}]]' ``` ``` tail -n 6 notifyOnNewRealTimeOverleafActivity.py.log ``` <details> <summary>Output:</summary> ``` 2025-10-19 18:11:32.605908 message='2::' 2025-10-19 18:11:50.161417 Checking at 1760890310.1613894 (lastTime=1760890292.6060472) 2025-10-19 18:11:57.633351 message='2::' 2025-10-19 18:12:15.161793 Checking at 1760890335.161772 (lastTime=1760890317.6334727) 2025-10-19 18:12:40.162049 Checking at 1760890360.162032 (lastTime=1760890317.6334727) 2025-10-19 18:13:05.162317 Checking at 1760890385.1622937 (lastTime=1760890317.6334727) ``` </details> It seems correct as I restarted yesterday OverClock3000.
Author
Owner

Next ping was expected at 2025-10-19 18:12:22.633351 which is quite exactly 24 hours after.

I am a bit surprised as I had the feeling that it was previously running for longer.

Benjamin-Loison/cpython/issues/29 would help, as the script is still running...

Next ping was expected at *2025-10-19 18:12:22.633351* which is quite exactly 24 hours after. I am a bit surprised as I had the feeling that it was previously running for longer. [Benjamin-Loison/cpython/issues/29](https://github.com/Benjamin-Loison/cpython/issues/29) would help, as the script is still running...
Author
Owner

The 24 hours duration is not due to page.wait_for_timeout, but this Python statement is maybe involved in not stopping the script.

The 24 hours duration is not due to `page.wait_for_timeout`, but this Python statement is maybe involved in not stopping the script.
Author
Owner

I restarted the script to see if again face the issue after 24 hours.

I restarted the script to see if again face the issue after 24 hours.
Author
Owner

I forgot to save the simplified script reproducing the non stopping behavior.

I forgot to save the simplified script reproducing the non stopping behavior.
Author
Owner

To be notified when someone leaves, as if do not write continuously can't know.

2025-10-20 22:40:34.362884 message='5:::{"name":"clientTracking.clientDisconnected","args":["P.XXXXXXXXXXXXXXXXXXXX"]}'

As possibly not multiple persons connected, just knowing that someone left if already nice.

People already connected when the script starts are sent as last_updated_at, see #issuecomment-4363.

args matches id, see #issuecomment-4343. Same if already connected with client_id.

Note that such id changes from a connection to the other for the same authenticated user.

Could maintain a dict to leverage the associated identity.

To be notified when someone leaves, as if do not write continuously can't know. ``` 2025-10-20 22:40:34.362884 message='5:::{"name":"clientTracking.clientDisconnected","args":["P.XXXXXXXXXXXXXXXXXXXX"]}' ``` As possibly not multiple persons connected, just knowing that someone left if already nice. People already connected when the script starts are sent as `last_updated_at`, see [#issuecomment-4363](#issuecomment-4363). `args` matches `id`, see [#issuecomment-4343](#issuecomment-4343). Same if already connected with `client_id`. Note that such id changes from a connection to the other for the same authenticated user. Could maintain a `dict` to leverage the associated identity.
Author
Owner

For the initialization:

print(json.dumps(json.loads(pyperclip.paste()), indent = 4))
Output:
[
    null,
    [
        {
            "last_updated_at": "1760992807670",
            "user_id": "XXXXXXXXXXXXXXXXXXXXXXXX",
            "first_name": "Benjamin",
            "last_name": "Loison",
            "email": "benjamin.loison@CENSORED.fr",
            "cursorData": {
                "row": CENSORED,
                "column": CENSORED,
                "doc_id": "XXXXXXXXXXXXXXXXXXXXXXXX"
            },
            "connected": true,
            "client_id": "P.XXXXXXXXXXXXXXXXXXXX",
            "client_age": 1.002
        },
        {
            "last_updated_at": "1760992807670",
            "user_id": "anonymous-user",
            "first_name": "",
            "last_name": "",
            "email": "",
            "connected": true,
            "client_id": "P.YYYYYYYYYYYYYYYYYYYY",
            "client_age": 1.003
        }
    ]
]
For the initialization: ```python print(json.dumps(json.loads(pyperclip.paste()), indent = 4)) ``` <details> <summary>Output:</summary> ```json [ null, [ { "last_updated_at": "1760992807670", "user_id": "XXXXXXXXXXXXXXXXXXXXXXXX", "first_name": "Benjamin", "last_name": "Loison", "email": "benjamin.loison@CENSORED.fr", "cursorData": { "row": CENSORED, "column": CENSORED, "doc_id": "XXXXXXXXXXXXXXXXXXXXXXXX" }, "connected": true, "client_id": "P.XXXXXXXXXXXXXXXXXXXX", "client_age": 1.002 }, { "last_updated_at": "1760992807670", "user_id": "anonymous-user", "first_name": "", "last_name": "", "email": "", "connected": true, "client_id": "P.YYYYYYYYYYYYYYYYYYYY", "client_age": 1.003 } ] ] ``` </details>
Author
Owner

Could merge {first,last}_name at initializion to have name if connected later.
Can first rely on email.

Could merge `{first,last}_name` at initializion to have `name` if connected later. Can first rely on `email`.
Author
Owner

Let us consider that Benjamin-Loison/matrix-commander/issues/16 may take multiple seconds and people may leave faster, as may have concurrency of him coming back for instance, well it would be using different ids.
So let us prefer keeping for too long memory content, but notify faster.

Let us consider that [Benjamin-Loison/matrix-commander/issues/16](https://github.com/Benjamin-Loison/matrix-commander/issues/16) may take multiple seconds and people may leave faster, as may have concurrency of him coming back for instance, well it would be using different ids. So let us prefer keeping for too long memory content, but notify _faster_.
Author
Owner

Note that if the person being connected when the script starts is DO_NOT_NOTIFY_ON_EMAIL, then it is not an issue.

Note that if the person being connected when the script starts is `DO_NOT_NOTIFY_ON_EMAIL`, then it is not an issue.
Sign in to join this conversation.
No Label
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Benjamin_Loison/overleaf#3
No description provided.