*curl: (3) URL rejected: Malformed input to a URL function* #8

Open
opened 2026-01-07 18:16:05 +01:00 by Benjamin_Loison · 2 comments
Owner

On my Linux Mint 22.2 Cinnamon Framework 13:

Bash script:
(
    curl https://www.wikidata.org/w/rest.php/wikibase/v1/entities/items/Q42/statements \ 
        -H 'Authorization: Bearer TEST'
) | sed -n 's/\(.\{25\}\).*\(.\{25\}\)/\1...\2/p'
Output:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  163k    0  163k    0     0   113k      0 --:--:--  0:00:01 --:--:--  113k
curl: (3) URL rejected: Malformed input to a URL function
-H: command not found
{"P31":[{"id":"Q42$F078E5...","content":"Q159288"}}]}

Note:

-H: command not found

do not have this issue if remove the newline.

DuckDuckGo and Google search Linux get first and last n characters and Linux cut from the end.

echo 0123456789 | cut -c 3- | rev | cut -c 3- | rev
234567

The Unix Stack Exchange answer 723446

echo 0123456789 | sed -n 's/\(.\{3\}\).*\(.\{3\}\)/\1...\2/p'
012...789

parentheses are necessary.

DuckDuckGo search sed p.

The Stack Overflow answer 20194559 helps to some extent precising what p stands for.

On my Linux Mint 22.2 Cinnamon Framework 13: <details> <summary>Bash script:</summary> ```bash ( curl https://www.wikidata.org/w/rest.php/wikibase/v1/entities/items/Q42/statements \ -H 'Authorization: Bearer TEST' ) | sed -n 's/\(.\{25\}\).*\(.\{25\}\)/\1...\2/p' ``` </details> <details> <summary>Output:</summary> ``` % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 163k 0 163k 0 0 113k 0 --:--:-- 0:00:01 --:--:-- 113k curl: (3) URL rejected: Malformed input to a URL function -H: command not found {"P31":[{"id":"Q42$F078E5...","content":"Q159288"}}]} ``` </details> Note: ``` -H: command not found ``` do not have this issue if remove the newline. DuckDuckGo and Google search *Linux get first and last n characters* and *Linux cut from the end*. ```bash echo 0123456789 | cut -c 3- | rev | cut -c 3- | rev ``` ``` 234567 ``` [The Unix Stack Exchange answer 723446](https://unix.stackexchange.com/a/723446) ```bash echo 0123456789 | sed -n 's/\(.\{3\}\).*\(.\{3\}\)/\1...\2/p' ``` ``` 012...789 ``` parentheses are necessary. DuckDuckGo search *sed p*. [The Stack Overflow answer 20194559](https://stackoverflow.com/a/20194559) helps to some extent precising what `p` stands for.
Author
Owner
Related to [Improve_websites_thanks_to_open_source/issues/3012](https://codeberg.org/Benjamin_Loison/Improve_websites_thanks_to_open_source/issues/3012).
Author
Owner

Related to Benjamin-Loison/android/issues/378.

curl 'https://example.com?a=b'
Output:
<!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content="width=device-width,initial-scale=1"><title>Example Domain</title><style>html{color-scheme:light dark;background:light-dark(#eee,#222)}body{font:16px/1.6 system-ui,sans-serif;max-width:26em;margin:auto;padding:25vh 2em 2em;text-align:center}</style></head><body><p>This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.</p><script src=/s.js></script></body></html>
curl 'https://example.com?a=b c'
curl: (3) URL rejected: Malformed input to a URL function

DuckDuckGo search curl with spaces.

Answers to the Stack Overflow question 5931853 do not help.

Are only spaces to blame in the mentioned Android issue?

Answers to the Super User question do not seem more general.

%20"Police%20municipale" seems not satisfying.

urlencode 'where=code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"'
code_insee_commune+LIKE+%22XXXXX%22+AND+nom+LIKE+%22Police+municipale%22

Firefox URL works fine:

https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"

curl "https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=$(urlencode 'code_insee_commune LIKE "78000" AND nom LIKE "Police municipale"')"
{"total_count": 0, "results": []}
Python script:
import requests
import json

URL = 'https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records'
PARAMETERS = {
    'where': 'code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"'
}
data = requests.get(URL, PARAMETERS).json()
print(json.dumps(data, indent = 4))

I was starting to wonder if should use + or %20.

curl -s "https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=$(urlencode 'code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"')" | jq -r .results[0].telephone | jq -r .[0].valeur
01 XX XX XX XX

Just using Code commune and not Code postal works as wanted.

Related to [Benjamin-Loison/android/issues/378](https://github.com/Benjamin-Loison/android/issues/378). ```bash curl 'https://example.com?a=b' ``` <details> <summary>Output:</summary> ```html <!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content="width=device-width,initial-scale=1"><title>Example Domain</title><style>html{color-scheme:light dark;background:light-dark(#eee,#222)}body{font:16px/1.6 system-ui,sans-serif;max-width:26em;margin:auto;padding:25vh 2em 2em;text-align:center}</style></head><body><p>This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.</p><script src=/s.js></script></body></html> ``` </details> ```bash curl 'https://example.com?a=b c' ``` ``` curl: (3) URL rejected: Malformed input to a URL function ``` DuckDuckGo search *curl with spaces*. Answers to [the Stack Overflow question 5931853](https://stackoverflow.com/q/5931853) do not help. Are only spaces to blame in the mentioned Android issue? Answers to [the Super User question](https://superuser.com/q/1374805) do not seem more general. `%20"Police%20municipale"` seems not satisfying. ```bash urlencode 'where=code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"' ``` ``` code_insee_commune+LIKE+%22XXXXX%22+AND+nom+LIKE+%22Police+municipale%22 ``` Firefox URL works fine: [https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"](https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=code_insee_commune%20LIKE%20%22XXXXX%22%20AND%20nom%20LIKE%20%22Police%20municipale%22) ```bash curl "https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=$(urlencode 'code_insee_commune LIKE "78000" AND nom LIKE "Police municipale"')" ``` ```json {"total_count": 0, "results": []} ``` <details> <summary>Python script:</summary> ```python import requests import json URL = 'https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records' PARAMETERS = { 'where': 'code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"' } data = requests.get(URL, PARAMETERS).json() print(json.dumps(data, indent = 4)) ``` </details> I was starting to wonder if should use `+` or `%20`. ```bash curl -s "https://api-lannuaire.service-public.gouv.fr/api/explore/v2.1/catalog/datasets/api-lannuaire-administration/records?where=$(urlencode 'code_insee_commune LIKE "XXXXX" AND nom LIKE "Police municipale"')" | jq -r .results[0].telephone | jq -r .[0].valeur ``` ``` 01 XX XX XX XX ``` Just using *Code commune* and not *Code postal* works as wanted.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Benjamin_Loison/curl#8