Benjamin_Loison/codeberg
1
0
Code Issues 25 Packages Projects Wiki

How many issue comments have I posted? #19

New Issue
Open
opened 2025-11-25 00:10:31 +01:00 by Benjamin_Loison · 14 comments
Benjamin_Loison commented 2025-11-25 00:10:31 +01:00
Owner
Copy Link
Copy Source

It is easy to see the number of issues with https://codeberg.org/issues.

https://blog.codeberg.org/letter-from-codeberg-onwards-and-upwards.html

more than 300k repositories

So I am 606 / 300,000, that is 1 / 495 of Codeberg repositories are mine.

Related to #1 and #13.

It is easy to see the number of issues with https://codeberg.org/issues. https://blog.codeberg.org/letter-from-codeberg-onwards-and-upwards.html > more than 300k repositories So I am 606 / 300,000, that is 1 / 495 of Codeberg repositories are mine. Related to #1 and #13.
Benjamin_Loison commented 2025-11-25 00:18:31 +01:00
Author
Owner
Copy Link
Copy Source

I verified that on repository with single issue with no comment (Benjamin_Loison/flathub-infra_website), listing comments does not return anything, but it works fine otherwise (Benjamin_Loison/android).

I verified that on repository with single issue with no comment ([Benjamin_Loison/flathub-infra_website](https://codeberg.org/Benjamin_Loison/flathub-infra_website)), listing comments does not return anything, but it works fine otherwise ([Benjamin_Loison/android](https://codeberg.org/Benjamin_Loison/android)).
Benjamin_Loison commented 2025-11-25 00:30:31 +01:00
Author
Owner
Copy Link
Copy Source
Python script: 
import requests
from tqdm import tqdm

# *Repository and Organization Access*: *All (public, private, and limited)*
# *Selection permissions*:
# - *issue*: *Read*
# - *repository*: *Read*
# - *user*: *Read*
TOKEN = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

HEADERS = {
    'Authorization': f'token {TOKEN}'
}

def getApi(url, params):
    return requests.get(f'https://codeberg.org/api/v1/{url}', params, headers = HEADERS).json()

def getItems(url, params):
    items = []
    with tqdm() as progressBar:
        while True:
            params['page'] = progressBar.n + 1
            data = getApi(url, params)
            items += data
            dataLen = len(data)
            if ('limit' in params and dataLen < params['limit']) or data == []:
                break
            progressBar.update(1)
    return items

PARAMS = {
    'limit': 50,
}

repos = getItems('user/repos', PARAMS)

totalIssueComments = 0
for repo in tqdm(repos, 'repo'):
    repoFullName = repo['full_name']
    comments = getItems(f'repos/{repoFullName}/issues/comments', {})
    totalIssueComments += len(comments)
    print(f'{totalIssueComments=}')
Output: 
...
totalIssueComments=16751
repo: 100%|██████████| 610/610 [05:33<00:00,  1.83it/s]
<details> <summary>Python script:</summary> ```python import requests from tqdm import tqdm # *Repository and Organization Access*: *All (public, private, and limited)* # *Selection permissions*: # - *issue*: *Read* # - *repository*: *Read* # - *user*: *Read* TOKEN = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' HEADERS = { 'Authorization': f'token {TOKEN}' } def getApi(url, params): return requests.get(f'https://codeberg.org/api/v1/{url}', params, headers = HEADERS).json() def getItems(url, params): items = [] with tqdm() as progressBar: while True: params['page'] = progressBar.n + 1 data = getApi(url, params) items += data dataLen = len(data) if ('limit' in params and dataLen < params['limit']) or data == []: break progressBar.update(1) return items PARAMS = { 'limit': 50, } repos = getItems('user/repos', PARAMS) totalIssueComments = 0 for repo in tqdm(repos, 'repo'): repoFullName = repo['full_name'] comments = getItems(f'repos/{repoFullName}/issues/comments', {}) totalIssueComments += len(comments) print(f'{totalIssueComments=}') ``` </details> <details> <summary>Output:</summary> ``` ... totalIssueComments=16751 repo: 100%|██████████| 610/610 [05:33<00:00, 1.83it/s] ``` </details>
Benjamin_Loison commented 2025-11-25 00:37:04 +01:00
Author
Owner
Copy Link
Copy Source

Let us consider my last issue comment Benjamin_Loison/server/issues/9#issuecomment-8456772.

16,751 / 8,456,772 = 1 / 504.9 of Codeberg issue comments are mine.

Benjamin_Loison/server/issues/9#issue-1089225

1,089,225 issues, so I am 7,474 + 455 = 7,929 of them, that is I am 1 / 137 of Codeberg issues.

Related to Benjamin-Loison/gitea/issues/57.

Let us consider my last issue comment [Benjamin_Loison/server/issues/9#issuecomment-8456772](https://codeberg.org/Benjamin_Loison/server/issues/9#issuecomment-8456772). 16,751 / 8,456,772 = 1 / 504.9 of Codeberg issue comments are mine. [Benjamin_Loison/server/issues/9#issue-1089225](https://codeberg.org/Benjamin_Loison/server/issues/9#issue-1089225) 1,089,225 issues, so I am 7,474 + 455 = 7,929 of them, that is I am 1 / 137 of Codeberg issues. Related to [Benjamin-Loison/gitea/issues/57](https://github.com/Benjamin-Loison/gitea/issues/57).
Benjamin_Loison commented 2025-11-25 00:39:24 +01:00
Author
Owner
Copy Link
Copy Source

In a nutshell:

On Codeberg:

  • 1 repository over 495
  • 1 issue over 137
  • 1 issue comment over 505

belong to me.

In a nutshell: On Codeberg: - 1 repository over 495 - 1 issue over 137 - 1 issue comment over 505 belong to me.
Benjamin_Loison commented 2025-11-25 00:40:26 +01:00
Author
Owner
Copy Link
Copy Source

Related to Benjamin-Loison/gitea/issues/46.

Related to [Benjamin-Loison/gitea/issues/46](https://github.com/Benjamin-Loison/gitea/issues/46).
Benjamin_Loison commented 2025-12-01 16:39:59 +01:00
Author
Owner
Copy Link
Copy Source

What about edits? Related to Benjamin_Loison/git/issues/53.

What about edits? Related to [Benjamin_Loison/git/issues/53](https://codeberg.org/Benjamin_Loison/git/issues/53).
Benjamin_Loison commented 2025-12-01 16:42:44 +01:00
Author
Owner
Copy Link
Copy Source

https://codeberg.org/Benjamin_Loison/xclip/issues/3/content-history/list

Output: 
{
  "results": [
    {
      "name": "<img loading=\"lazy\" alt=\"\" class=\"ui avatar tw-align-middle tw-mr-2\" src=\"/user/avatar/Benjamin_Loison/0\" title=\"Benjamin_Loison\" width=\"28\" height=\"28\"/><strong>Benjamin_Loison</strong> edited <relative-time prefix=\"\" tense=\"past\" datetime=\"2025-11-28T11:53:19+01:00\" data-tooltip-content data-tooltip-interactive=\"true\">2025-11-28 11:53:19 +01:00</relative-time>",
      "value": 1433628
    },
    {
      "name": "<img loading=\"lazy\" alt=\"\" class=\"ui avatar tw-align-middle tw-mr-2\" src=\"/user/avatar/Benjamin_Loison/0\" title=\"Benjamin_Loison\" width=\"28\" height=\"28\"/><strong>Benjamin_Loison</strong> created <relative-time prefix=\"\" tense=\"past\" datetime=\"2025-11-28T11:52:42+01:00\" data-tooltip-content data-tooltip-interactive=\"true\">2025-11-28 11:52:42 +01:00</relative-time>",
      "value": 1433625
    }
  ]
}

Except if need to add an edit to count the initial issue comment one, there is less, if it is value, edits than issue.

Benjamin_Loison/xclip/issues/3#issue-2769423

https://codeberg.org/Benjamin_Loison/xclip/issues/3/content-history/list <details> <summary>Output:</summary> ```json { "results": [ { "name": "<img loading=\"lazy\" alt=\"\" class=\"ui avatar tw-align-middle tw-mr-2\" src=\"/user/avatar/Benjamin_Loison/0\" title=\"Benjamin_Loison\" width=\"28\" height=\"28\"/><strong>Benjamin_Loison</strong> edited <relative-time prefix=\"\" tense=\"past\" datetime=\"2025-11-28T11:53:19+01:00\" data-tooltip-content data-tooltip-interactive=\"true\">2025-11-28 11:53:19 +01:00</relative-time>", "value": 1433628 }, { "name": "<img loading=\"lazy\" alt=\"\" class=\"ui avatar tw-align-middle tw-mr-2\" src=\"/user/avatar/Benjamin_Loison/0\" title=\"Benjamin_Loison\" width=\"28\" height=\"28\"/><strong>Benjamin_Loison</strong> created <relative-time prefix=\"\" tense=\"past\" datetime=\"2025-11-28T11:52:42+01:00\" data-tooltip-content data-tooltip-interactive=\"true\">2025-11-28 11:52:42 +01:00</relative-time>", "value": 1433625 } ] } ``` </details> Except if need to add an edit to count the initial issue comment one, there is less, if it is `value`, edits than issue. [Benjamin_Loison/xclip/issues/3#issue-2769423](https://codeberg.org/Benjamin_Loison/xclip/issues/3#issue-2769423)
Benjamin_Loison commented 2025-12-03 10:33:56 +01:00
Author
Owner
Copy Link
Copy Source

Related to Benjamin-Loison/gitea/issues/87.

Related to [Benjamin-Loison/gitea/issues/87](https://github.com/Benjamin-Loison/gitea/issues/87).
Benjamin_Loison commented 2025-12-03 16:54:36 +01:00
Author
Owner
Copy Link
Copy Source

https://codeberg.org/Benjamin_Loison/xclip/issues/4/content-history/list with no edit and issue comment {"results":null}.

https://codeberg.org/api/swagger#/issue revision and edit:

Benjamin_Loison/xclip/issues/5

https://codeberg.org/Benjamin_Loison/xclip/issues/5/content-history/overview:

Output: 
{
  "editedHistoryCountMap": {
    "8670009": 2
  },
  "i18n": {
    "textDeleteFromHistory": "Delete from history",
    "textDeleteFromHistoryConfirm": "Delete from history?",
    "textEdited": "edited",
    "textOptions": "Options"
  }
}

Benjamin_Loison/xclip/issues/5#issuecomment-8670009 is the issue comment I let to the issue.

Output: 
{
  "editedHistoryCountMap": {
    "8670009": 3
  },
  "i18n": {
    "textDeleteFromHistory": "Delete from history",
    "textDeleteFromHistoryConfirm": "Delete from history?",
    "textEdited": "edited",
    "textOptions": "Options"
  }
}

after having edited the second issue comment.

If edit the initial issue comment: 
{
  "editedHistoryCountMap": {
    "0": 2,
    "8670009": 3
  },
  "i18n": {
    "textDeleteFromHistory": "Delete from history",
    "textDeleteFromHistoryConfirm": "Delete from history?",
    "textEdited": "edited",
    "textOptions": "Options"
  }
}

So if want to count edits, then have to sum these values, each minus 1.

https://codeberg.org/Benjamin_Loison/xclip/issues/4/content-history/list with no edit and issue comment `{"results":null}`. https://codeberg.org/api/swagger#/issue *revision* and *edit*: [Benjamin_Loison/xclip/issues/5](https://codeberg.org/Benjamin_Loison/xclip/issues/5) https://codeberg.org/Benjamin_Loison/xclip/issues/5/content-history/overview: <details> <summary>Output:</summary> ```json { "editedHistoryCountMap": { "8670009": 2 }, "i18n": { "textDeleteFromHistory": "Delete from history", "textDeleteFromHistoryConfirm": "Delete from history?", "textEdited": "edited", "textOptions": "Options" } } ``` </details> [Benjamin_Loison/xclip/issues/5#issuecomment-8670009](https://codeberg.org/Benjamin_Loison/xclip/issues/5#issuecomment-8670009) is the issue comment I let to the issue. <details> <summary>Output:</summary> ```json { "editedHistoryCountMap": { "8670009": 3 }, "i18n": { "textDeleteFromHistory": "Delete from history", "textDeleteFromHistoryConfirm": "Delete from history?", "textEdited": "edited", "textOptions": "Options" } } ``` </details> after having edited the second issue comment. <details> <summary>If edit the initial issue comment:</summary> ```json { "editedHistoryCountMap": { "0": 2, "8670009": 3 }, "i18n": { "textDeleteFromHistory": "Delete from history", "textDeleteFromHistoryConfirm": "Delete from history?", "textEdited": "edited", "textOptions": "Options" } } ``` </details> So if want to count edits, then have to sum these values, each minus 1.
Benjamin_Loison commented 2025-12-03 17:23:29 +01:00
Author
Owner
Copy Link
Copy Source

https://codeberg.org/api/swagger#/user/userGetCurrent
https://codeberg.org/api/swagger#/repository/repoGet
https://codeberg.org/api/swagger#/issue/issueListIssues
https://codeberg.org/api/swagger#/issue/issueGetRepoComments
https://codeberg.org/api/swagger#/issue/issueGetIssue

do not help.

https://codeberg.org/api/swagger#/issue/issueGetComment

https://codeberg.org/api/v1/repos/Benjamin_Loison/xclip/issues/comments/2769423?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Output: 
{
  "message": "The target couldn't be found.",
  "url": "https://codeberg.org/api/swagger",
  "errors": [
    "comment does not exist [id: 2769423, issue_id: 0]"
  ]
}

https://codeberg.org/api/v1/repos/Benjamin_Loison/xclip/issues/comments/1?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Output: 
{
  "message": "The target couldn't be found.",
  "url": "https://codeberg.org/api/swagger",
  "errors": []
}

unclear how to specify issue_id instead of id.

Error: 
Traceback (most recent call last):
  File "<tmp 1>", line 42, in <module>
    contentHistory = requests.get(f'https://codeberg.org/{repoFullName}/issues/{issue["number"]}/content-history/overview').json()
                                                                                ~~~~~^^^^^^^^^^
KeyError: 'number'
Output: 
{
    "id": 2501237,
    "html_url": "https://codeberg.org/Benjamin_Loison/7zip/issues/3#issuecomment-2501237",
    "pull_request_url": "",
    "issue_url": "https://codeberg.org/Benjamin_Loison/7zip/issues/3",
    "user": {
        "id": 74932,
        "login": "Benjamin_Loison",
        "login_name": "",
        "source_id": 0,
        "full_name": "Benjamin Loison",
        "email": "benjamin_loison@noreply.codeberg.org",
        "avatar_url": "https://codeberg.org/avatars/1920d1f51a3ae1c236dda41e05b2b592debb3635ce9a89f9bc22d291c9ba62fc",
        "html_url": "https://codeberg.org/Benjamin_Loison",
        "language": "",
        "is_admin": false,
        "last_login": "0001-01-01T00:00:00Z",
        "created": "2022-12-05T16:13:33+01:00",
        "restricted": false,
        "active": false,
        "prohibit_login": false,
        "location": "",
        "pronouns": "",
        "website": "",
        "description": "",
        "visibility": "public",
        "followers_count": 1,
        "following_count": 1,
        "starred_repos_count": 4,
        "username": "Benjamin_Loison"
    },
    "original_author": "",
    "original_author_id": 0,
    "body": "Related to [Benjamin_Loison/file-roller/issues/7](https://gitlab.gnome.org/Benjamin_Loison/file-roller/-/issues/7).",
    "assets": [],
    "created_at": "2024-12-07T16:03:48+01:00",
    "updated_at": "2024-12-07T16:03:48+01:00"
}

Related to Benjamin-Loison/gitea/issues/220.

https://codeberg.org/api/swagger#/user/userGetCurrent https://codeberg.org/api/swagger#/repository/repoGet https://codeberg.org/api/swagger#/issue/issueListIssues https://codeberg.org/api/swagger#/issue/issueGetRepoComments https://codeberg.org/api/swagger#/issue/issueGetIssue do not help. https://codeberg.org/api/swagger#/issue/issueGetComment https://codeberg.org/api/v1/repos/Benjamin_Loison/xclip/issues/comments/2769423?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX <details> <summary>Output:</summary> ```json { "message": "The target couldn't be found.", "url": "https://codeberg.org/api/swagger", "errors": [ "comment does not exist [id: 2769423, issue_id: 0]" ] } ``` </details> https://codeberg.org/api/v1/repos/Benjamin_Loison/xclip/issues/comments/1?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX <details> <summary>Output:</summary> ```json { "message": "The target couldn't be found.", "url": "https://codeberg.org/api/swagger", "errors": [] } ``` </details> unclear how to specify `issue_id` instead of `id`. <details> <summary>Error:</summary> ``` Traceback (most recent call last): File "<tmp 1>", line 42, in <module> contentHistory = requests.get(f'https://codeberg.org/{repoFullName}/issues/{issue["number"]}/content-history/overview').json() ~~~~~^^^^^^^^^^ KeyError: 'number' ``` </details> <details> <summary>Output:</summary> ```json { "id": 2501237, "html_url": "https://codeberg.org/Benjamin_Loison/7zip/issues/3#issuecomment-2501237", "pull_request_url": "", "issue_url": "https://codeberg.org/Benjamin_Loison/7zip/issues/3", "user": { "id": 74932, "login": "Benjamin_Loison", "login_name": "", "source_id": 0, "full_name": "Benjamin Loison", "email": "benjamin_loison@noreply.codeberg.org", "avatar_url": "https://codeberg.org/avatars/1920d1f51a3ae1c236dda41e05b2b592debb3635ce9a89f9bc22d291c9ba62fc", "html_url": "https://codeberg.org/Benjamin_Loison", "language": "", "is_admin": false, "last_login": "0001-01-01T00:00:00Z", "created": "2022-12-05T16:13:33+01:00", "restricted": false, "active": false, "prohibit_login": false, "location": "", "pronouns": "", "website": "", "description": "", "visibility": "public", "followers_count": 1, "following_count": 1, "starred_repos_count": 4, "username": "Benjamin_Loison" }, "original_author": "", "original_author_id": 0, "body": "Related to [Benjamin_Loison/file-roller/issues/7](https://gitlab.gnome.org/Benjamin_Loison/file-roller/-/issues/7).", "assets": [], "created_at": "2024-12-07T16:03:48+01:00", "updated_at": "2024-12-07T16:03:48+01:00" } ``` </details> Related to [Benjamin-Loison/gitea/issues/220](https://github.com/Benjamin-Loison/gitea/issues/220).
Benjamin_Loison commented 2025-12-03 17:52:30 +01:00
Author
Owner
Copy Link
Copy Source

With private repository get Not found.\n.

minimizeCURL curl.sh '1099127'
Output: 
Initial command length: 785.
Removing headers
Command with length 695 is still fine.
Command with length 678 is still fine.
Command with length 641 is still fine.
Command with length 595 is still fine.
Command with length 567 is still fine.
Command with length 539 is still fine.
Command with length 512 is still fine.
Command with length 486 is still fine.
Command with length 453 is still fine.
Command with length 434 is still fine.
Command with length 416 is still fine.
Removing URL parameters
Removing cookies
Command with length 266 is still fine.
Command with length 204 is still fine.
Command with length 192 is still fine.
Command with length 161 is still fine.
Command with length 135 is still fine.
Removing raw data
curl https://codeberg.org/Benjamin_Loison/CENSORED/issues/3/content-history/overview -H 'Cookie: cb_sessionid=XXXXXXXXXXXXXXXX'
Error: 
repo:  21%|███████████████████▉                                                                          | 132/622 [11:46<43:42,  5.35s/it]
Traceback (most recent call last):
  File "/home/benjamin_loison/gitea.py", line 49, in <module>
    contentHistory = json.loads(text)#.json()
  File "/usr/lib/python3.13/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/usr/lib/python3.13/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: 
<!doctype html>
<title>429 Too many requests - Codeberg.org</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
	body { text-align: center; font: 20px Helvetica, sans-serif; color: #333; }
h1 { font-size: 3rem; }
article { display: block; text-align: start; margin: 0 auto; }
a { color: #2185d0; text-decoration: none; }
a:hover { text-decoration: none; }
@media (prefers-color-scheme: dark) {
	body { background-color: #171e26; color: #fff; }
	a { color: #4793cc; }
}
@media only screen and (min-width: 48rem) {
	body { padding: 9rem; }
	article { width: 40rem; }
}

</style>

<article>
	<h1>Too many requests</h1>
	<p>Your connection is temporarily throttled, because you sent too many requests to Codeberg. This could have various reasons:.</p>
	<ul>
		<li>You have caused some traffic to expensive routes at Codeberg, such as viewing Git blames or comparing large commits.</li>
		<li>Your traffic looked like crawling, e.g. by heavily browsing code at specific commit timestamps.</li>
		<li>We are currently working on our systems and might have made a configuration mistake.</li>
	</ul>
	<p>We are sorry for the inconvenience. Please wait for 15 - 30 minutes and try again. If you keep running into this, please let us know and describe what you were doing.</p>
	
	<p>
		If this error continues to show up, please have a look at our <a href="https://status.codeberg.org">status page</a>.<br>
	</p>
</article>
Python script: 
import requests
from tqdm import tqdm
import json
from lxml import html
import time

# *Repository and Organization Access*: *All (public, private, and limited)*
# *Selection permissions*:
# - *issue*: *Read*
# - *repository*: *Read*
# - *user*: *Read*
TOKEN = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

HEADERS = {
    'Authorization': f'token {TOKEN}'
}

#secondsToWait = 1

def getApiUrl(url):
    return f'https://codeberg.org/api/v1/{url}'

def requestGetText(url, params, headers, cookies):
    secondsToWait = 1
    while True:
        try:
            return requests.get(url, params = params, headers = headers, cookies = cookies).text
        except:
            secondsToWait *= 2
            print(f'Request sleeping {secondsToWait} seconds...')
            time.sleep(secondsToWait)
            print(f'Request slept {secondsToWait} seconds!')

def requestGetJson(url, params, headers, cookies):
    secondsToWait = 1
    while True:
        text = requestGetText(url, params, headers, cookies)
        try:
            #return json.loads(text)
            #print('a')
            data = json.loads(text)
            #print('b')
            return data
        except json.decoder.JSONDecodeError:
            if getHtml(text).xpath('//title')[0].text == '429 Too many requests - Codeberg.org':
                secondsToWait *= 2
            else:
                print(f'{text=}')
                exit(1)
        print(f'Codeberg sleeping {secondsToWait} seconds...')
        time.sleep(secondsToWait)
        print(f'Codeberg slept {secondsToWait} seconds!')

def getHtml(text):
    return html.fromstring(text)

def getApi(url, params):
    return requestGetJson(url, params, HEADERS, {})
    '''
    text = requestGetJson(url, params, HEADERS, {})
    try:
        return json.loads(text)
    except:
        print(f'{text=}')
        exit(1)
    '''

def getItems(url, params):
    url = getApiUrl(url)
    items = []
    with tqdm() as progressBar:
        while True:
            params['page'] = progressBar.n + 1
            data = getApi(url, params)
            items += data
            dataLen = len(data)
            if ('limit' in params and dataLen < params['limit']) or data == []:
                break
            progressBar.update(1)
    return items

PARAMS = {
    'limit': 50,
}

repos = getItems('user/repos', PARAMS)

COOKIES = {
    'cb_sessionid': 'XXXXXXXXXXXXXXXX',
}

totalIssueEdits = 0
for repo in tqdm(repos, 'repo'):
    repoFullName = repo['full_name']
    issues = getItems(f'repos/{repoFullName}/issues/comments', {})
    for issue in tqdm(issues, 'issue'):
        issueNumber = issue['issue_url'].split('/')[-1] #issue['number']
        contentHistory = requestGetJson(f'https://codeberg.org/{repoFullName}/issues/{issueNumber}/content-history/overview', {}, {}, COOKIES)
        editedHistoryCountMap = contentHistory['editedHistoryCountMap']
        totalIssueEdits += sum(editedHistoryCountMap.values()) - len(editedHistoryCountMap)
    print(f'{totalIssueEdits=}')
With private repository get `Not found.\n`. ```bash minimizeCURL curl.sh '1099127' ``` <details> <summary>Output:</summary> ``` Initial command length: 785. Removing headers Command with length 695 is still fine. Command with length 678 is still fine. Command with length 641 is still fine. Command with length 595 is still fine. Command with length 567 is still fine. Command with length 539 is still fine. Command with length 512 is still fine. Command with length 486 is still fine. Command with length 453 is still fine. Command with length 434 is still fine. Command with length 416 is still fine. Removing URL parameters Removing cookies Command with length 266 is still fine. Command with length 204 is still fine. Command with length 192 is still fine. Command with length 161 is still fine. Command with length 135 is still fine. Removing raw data curl https://codeberg.org/Benjamin_Loison/CENSORED/issues/3/content-history/overview -H 'Cookie: cb_sessionid=XXXXXXXXXXXXXXXX' ``` </details> <details> <summary>Error:</summary> ``` repo: 21%|███████████████████▉ | 132/622 [11:46<43:42, 5.35s/it] Traceback (most recent call last): File "/home/benjamin_loison/gitea.py", line 49, in <module> contentHistory = json.loads(text)#.json() File "/usr/lib/python3.13/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ~~~~~~~~~~~~~~~~~~~~~~~^^^ File "/usr/lib/python3.13/json/decoder.py", line 345, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/json/decoder.py", line 363, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` </details> <details> <summary>Error:</summary> ```html <!doctype html> <title>429 Too many requests - Codeberg.org</title> <meta name="viewport" content="width=device-width, initial-scale=1"> <style> body { text-align: center; font: 20px Helvetica, sans-serif; color: #333; } h1 { font-size: 3rem; } article { display: block; text-align: start; margin: 0 auto; } a { color: #2185d0; text-decoration: none; } a:hover { text-decoration: none; } @media (prefers-color-scheme: dark) { body { background-color: #171e26; color: #fff; } a { color: #4793cc; } } @media only screen and (min-width: 48rem) { body { padding: 9rem; } article { width: 40rem; } } </style> <article> <h1>Too many requests</h1> <p>Your connection is temporarily throttled, because you sent too many requests to Codeberg. This could have various reasons:.</p> <ul> <li>You have caused some traffic to expensive routes at Codeberg, such as viewing Git blames or comparing large commits.</li> <li>Your traffic looked like crawling, e.g. by heavily browsing code at specific commit timestamps.</li> <li>We are currently working on our systems and might have made a configuration mistake.</li> </ul> <p>We are sorry for the inconvenience. Please wait for 15 - 30 minutes and try again. If you keep running into this, please let us know and describe what you were doing.</p> <p> If this error continues to show up, please have a look at our <a href="https://status.codeberg.org">status page</a>.<br> </p> </article> ``` </details> <details> <summary>Python script:</summary> ```python import requests from tqdm import tqdm import json from lxml import html import time # *Repository and Organization Access*: *All (public, private, and limited)* # *Selection permissions*: # - *issue*: *Read* # - *repository*: *Read* # - *user*: *Read* TOKEN = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' HEADERS = { 'Authorization': f'token {TOKEN}' } #secondsToWait = 1 def getApiUrl(url): return f'https://codeberg.org/api/v1/{url}' def requestGetText(url, params, headers, cookies): secondsToWait = 1 while True: try: return requests.get(url, params = params, headers = headers, cookies = cookies).text except: secondsToWait *= 2 print(f'Request sleeping {secondsToWait} seconds...') time.sleep(secondsToWait) print(f'Request slept {secondsToWait} seconds!') def requestGetJson(url, params, headers, cookies): secondsToWait = 1 while True: text = requestGetText(url, params, headers, cookies) try: #return json.loads(text) #print('a') data = json.loads(text) #print('b') return data except json.decoder.JSONDecodeError: if getHtml(text).xpath('//title')[0].text == '429 Too many requests - Codeberg.org': secondsToWait *= 2 else: print(f'{text=}') exit(1) print(f'Codeberg sleeping {secondsToWait} seconds...') time.sleep(secondsToWait) print(f'Codeberg slept {secondsToWait} seconds!') def getHtml(text): return html.fromstring(text) def getApi(url, params): return requestGetJson(url, params, HEADERS, {}) ''' text = requestGetJson(url, params, HEADERS, {}) try: return json.loads(text) except: print(f'{text=}') exit(1) ''' def getItems(url, params): url = getApiUrl(url) items = [] with tqdm() as progressBar: while True: params['page'] = progressBar.n + 1 data = getApi(url, params) items += data dataLen = len(data) if ('limit' in params and dataLen < params['limit']) or data == []: break progressBar.update(1) return items PARAMS = { 'limit': 50, } repos = getItems('user/repos', PARAMS) COOKIES = { 'cb_sessionid': 'XXXXXXXXXXXXXXXX', } totalIssueEdits = 0 for repo in tqdm(repos, 'repo'): repoFullName = repo['full_name'] issues = getItems(f'repos/{repoFullName}/issues/comments', {}) for issue in tqdm(issues, 'issue'): issueNumber = issue['issue_url'].split('/')[-1] #issue['number'] contentHistory = requestGetJson(f'https://codeberg.org/{repoFullName}/issues/{issueNumber}/content-history/overview', {}, {}, COOKIES) editedHistoryCountMap = contentHistory['editedHistoryCountMap'] totalIssueEdits += sum(editedHistoryCountMap.values()) - len(editedHistoryCountMap) print(f'{totalIssueEdits=}') ``` </details>
Benjamin_Loison commented 2025-12-03 19:04:10 +01:00
Author
Owner
Copy Link
Copy Source

Could assume that only the UI faces this issue, not the API.

Could assume that only the UI faces this issue, not the API.
Benjamin_Loison commented 2025-12-03 19:30:35 +01:00
Author
Owner
Copy Link
Copy Source
Output: 
...
repo:  16%|█▌        | 99/622 [08:19<43:58,  5.05s/it]
Traceback (most recent call last):
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/socket.py", line 977, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 488, in _make_request
    raise new_e
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 464, in _make_request
    self._validate_conn(conn)
    ~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
    conn.connect()
    ~~~~~~~~~~~~^^
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 753, in connect
    self.sock = sock = self._new_conn()
                       ~~~~~~~~~~~~~~^^
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 205, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/adapters.py", line 644, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='codeberg.org', port=443): Max retries exceeded with url: /Benjamin_Loison/curl_cffi/issues/3/content-history/overview (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/benjamin_loison/gitea.py", line 90, in <module>
    contentHistory = requestGetJson(f'https://codeberg.org/{repoFullName}/issues/{issueNumber}/content-history/overview', {}, {}, COOKIES)
  File "/home/benjamin_loison/gitea.py", line 29, in requestGetJson
    text = requestGetText(url, params, headers, cookies)
  File "/home/benjamin_loison/gitea.py", line 24, in requestGetText
    return requests.get(url, params = params, headers = headers, cookies = cookies).text
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/adapters.py", line 677, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='codeberg.org', port=443): Max retries exceeded with url: /Benjamin_Loison/curl_cffi/issues/3/content-history/overview (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known)"))
<details> <summary>Output:</summary> ``` ... repo: 16%|█▌ | 99/622 [08:19<43:58, 5.05s/it] Traceback (most recent call last): File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( (self._dns_host, self.port), ...<2 lines>... socket_options=self.socket_options, ) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/util/connection.py", line 60, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.13/socket.py", line 977, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ socket.gaierror: [Errno -2] Name or service not known The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 787, in urlopen response = self._make_request( conn, ...<10 lines>... **response_kw, ) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 488, in _make_request raise new_e File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 464, in _make_request self._validate_conn(conn) ~~~~~~~~~~~~~~~~~~~^^^^^^ File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn conn.connect() ~~~~~~~~~~~~^^ File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 753, in connect self.sock = sock = self._new_conn() ~~~~~~~~~~~~~~^^ File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connection.py", line 205, in _new_conn raise NameResolutionError(self.host, self, e) from e urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/adapters.py", line 644, in send resp = conn.urlopen( method=request.method, ...<9 lines>... chunked=chunked, ) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/connectionpool.py", line 841, in urlopen retries = retries.increment( method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] ) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='codeberg.org', port=443): Max retries exceeded with url: /Benjamin_Loison/curl_cffi/issues/3/content-history/overview (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known)")) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/benjamin_loison/gitea.py", line 90, in <module> contentHistory = requestGetJson(f'https://codeberg.org/{repoFullName}/issues/{issueNumber}/content-history/overview', {}, {}, COOKIES) File "/home/benjamin_loison/gitea.py", line 29, in requestGetJson text = requestGetText(url, params, headers, cookies) File "/home/benjamin_loison/gitea.py", line 24, in requestGetText return requests.get(url, params = params, headers = headers, cookies = cookies).text ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, **kwargs) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/home/benjamin_loison/venv/lib/python3.13/site-packages/requests/adapters.py", line 677, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='codeberg.org', port=443): Max retries exceeded with url: /Benjamin_Loison/curl_cffi/issues/3/content-history/overview (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f8e5b0aefd0>: Failed to resolve 'codeberg.org' ([Errno -2] Name or service not known)")) ``` </details>
Benjamin_Loison commented 2025-12-03 22:13:24 +01:00
Author
Owner
Copy Link
Copy Source
totalIssueEdits=59139

stat logs.txt
Output: 
  File: logs.txt
  Size: 7795            Blocks: 16         IO Block: 4096   regular file
Device: 253,1   Inode: 31719506    Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/benjamin_loison)   Gid: ( 1000/benjamin_loison)
Access: 2025-12-03 22:11:55.991817058 +0100
Modify: 2025-12-03 19:54:46.864018977 +0100
Change: 2025-12-03 19:54:46.864018977 +0100
 Birth: 2025-12-03 19:00:51.712098358 +0100

So took about 43 minutes.

``` totalIssueEdits=59139 ``` ``` stat logs.txt ``` <details> <summary>Output:</summary> ``` File: logs.txt Size: 7795 Blocks: 16 IO Block: 4096 regular file Device: 253,1 Inode: 31719506 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 1000/benjamin_loison) Gid: ( 1000/benjamin_loison) Access: 2025-12-03 22:11:55.991817058 +0100 Modify: 2025-12-03 19:54:46.864018977 +0100 Change: 2025-12-03 19:54:46.864018977 +0100 Birth: 2025-12-03 19:00:51.712098358 +0100 ``` </details> So took about 43 minutes.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
Benjamin_Loison (Benjamin Loison)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Benjamin_Loison/codeberg#19