Akash Mahanty
b79f1c471e
Merge pull request #135 from eggplants/fix_cli
...
Fix cli.py
2022-02-05 16:54:36 +05:30
eggplants
d2a3946425
fix: escape banner
2022-02-05 10:12:27 +09:00
eggplants
7b6401d59b
fix: delete useless conds
2022-02-05 06:20:03 +09:00
eggplants
ed6160c54f
add: TooManyRequestsError
2022-02-05 06:19:02 +09:00
eggplants
fcab19a40a
fix: cli
...
print error message to stderr and specify defaults of url
2022-02-05 05:55:04 +09:00
eggplants
5f3cd28046
Fix Pylint errors were pointed out by codacy ( #133 )
...
* fix: pylint errors were pointed out by codacy
* fix: line length
* fix: help text
* fix: revert
https://stackoverflow.com/a/64477857 makes cli unusable
* fix: cli error and refactor codes
2022-02-05 05:25:40 +09:00
Akash Mahanty
b69e4dff37
rename params of main in cli.py to avoid using built-ins ( #132 )
...
* rename params of main in cli.py to avoid using built-ins
* Fix Line 32:80: E501 line too long (102 > 79 characters)
2022-02-05 00:30:35 +05:30
eggplants
d8cabdfdb5
Typing ( #128 )
...
* fix: CI yml name
* add: mypy configuraion
* add: type annotation to waybackpy modules
* add: type annotation to test modules
* fix: mypy command
* add: types-requests to dev deps
* fix: disable max-line-length
* fix: move pytest.ini into setup.cfg
* add: urllib3 to deps
* fix: Retry (ref: https://github.com/python/typeshed/issues/6893 )
* fix: f-string
* fix: shorten long lines
* add: staticmethod decorator to no-self-use methods
* fix: str(headers)->headers_str
* fix: error message
* fix: revert "str(headers)->headers_str" and ignore assignment CaseInsensitiveDict with str
* fix: mypy error
2022-02-05 03:23:36 +09:00
eggplants
e61447effd
Format and lint codes and fix packaging ( #125 )
...
* add: configure files (setup.py->setup.py+setup.cfg+pyproject.toml)
* add: __download_url__
* format with black and isort
* fix: flake8 section in setup.cfg
* add: E501 to flake ignore
* fix: metadata.name does not accept attr
* fix: merge __version__.py into __init__.py
* fix: flake8 errors in tests/
* fix: datetime.datetime -> datetime
* fix: banner
* fix: ignore W605 for banner
* fix: way to install deps in CI
* add: versem to setuptools
* fix: drop python<=3.6 (#126 ) from package and CI
2022-02-03 19:13:39 +05:30
eggplants
bc1dc4dc96
fix: save retry mechanism
2022-02-03 19:45:16 +09:00
Akash Mahanty
5cbdfc040b
waybackpy/cli.py : remove duplicate original_string from output_string in cdx
2022-01-30 21:02:25 +05:30
Akash Mahanty
946c28eddf
waybackpy/cli.py: Added help text, fix bug in the cdx_print parameter and lots of other stuff
...
parameter --filters is now --filter
parameter --collapses is now --collapse
added a new --license flag for fetching the license from GitHub repo and printing it.
2022-01-30 20:00:50 +05:30
Akash Mahanty
004027f73b
waybackpy/utils.py : Add a new function(latest_version_github) to fetch the latest release from github api and renamed latest_version to latest_version_pypi as now we have two functions to get the latest release.
2022-01-30 13:28:13 +05:30
Akash Mahanty
9dbe3b3bf4
In waybackpy/wrapper.py set self.timestamp to None on init.
...
In older interface(2.x.x) we had timestamp set to none in the constructer, so maybe it should be best to set it to None in the backwards compatiblliy module.)
2022-01-29 22:12:02 +05:30
Akash Mahanty
f03b2cb6cb
fix formatting of ASCII art
2022-01-26 18:24:24 +05:30
Akash Mahanty
5e0ea023e6
update CLI help text
2022-01-26 16:23:24 +05:30
Akash Mahanty
9007149fef
3.0.1 -- > 3.0.2, for condaforge staged-recipes issues
2022-01-26 01:54:58 +05:30
Akash Mahanty
5ea1d3ba4f
Replace NON-ASCII character figlet with ASCII character figlet.
2022-01-26 01:46:42 +05:30
Akash Mahanty
d79b10c74c
v3.0.0 --> v3.0.1
2022-01-25 19:52:10 +05:30
Akash Mahanty
06095202fe
BUG FIX : forgot to use the endpoint from the instance and also assign payload to param. Bug caught by the flake8 in the CI tests.
2022-01-24 23:35:48 +05:30
Akash Mahanty
06fc7855bf
waybackpy/cdx_api.py : deafult user agent is now DEFAULT_USER_AGENT, get_response now take url and headers as arguments and request url is generated by full_url function. max_tries added as parameter for the WaybackMachineCDXServerAPI class with default value of 3.
2022-01-24 23:20:49 +05:30
Akash Mahanty
c49fe971fd
update the older deprecation not for Url class, the newer date is now 2025 instead of 2024.
2022-01-24 23:15:59 +05:30
Akash Mahanty
9262f5da21
improve functions get_total_pages, get_response and lint check_filters, check_collapses and check_match_type
...
get_total_pages : default user agent is now DEFAULT_USER_AGENT
and now instead of str formatting passing payload
as param to full_url to generate the request url
also get_response make the request instead of directly
using requests.get()
get_response : get_response is now not taking param as keyword arguments
instead the invoker is supposed to pass the full url which
may be generated by the full_url function therefore the return_full_url=False,
is deprecated also.
Also now closing the session via session.close()
No need to check 'Exceeded 30 redirects' as save API uses a
diffrent method.
check_filters : Not assigning to variables the return of match groups
beacause we wont be using them and the linter picks these
unused assignments.
check_collapses : Same reason as for check_filters but also removed a foolish
test that checks equality with objects that are guaranteed
to be same.
check_match_type : Updated the text that of WaybackError
2022-01-24 22:57:20 +05:30
Akash Mahanty
d1a1cf2546
added tests for utils.py at tests/test_utils.py also changed a keyword argument from headers to user_agent for latest_version of utils.py with the usual default vaule.
2022-01-24 17:50:36 +05:30
Akash Mahanty
cd8a32ed1f
added tests for cdx_snapshot.py at tests/test_cdx_snapshot.py
2022-01-24 16:29:44 +05:30
Akash Mahanty
2bea92b348
fix bug with the third matching case of the archive_url_parser, caught while writing more tests fo the save API interface.
2022-01-24 13:31:30 +05:30
Akash Mahanty
7844d15d99
close the session in save api interface
2022-01-23 18:34:06 +05:30
Akash Mahanty
c0252edff2
updated tests for availability_api.py and also added max_tries(default value is 3) with delay (sleep) between successive API calls. The dealy actually improves the performace of the availability_api interface.
2022-01-23 15:05:10 +05:30
Akash Mahanty
1bacd73002
created pytest.ini, added test for waybackpy/availability_api.py, new exceptions all of which inherit from the main WaybackError and created requirements-dev.txt
2022-01-23 01:29:07 +05:30
Akash Mahanty
79901ba968
updated README.md
2022-01-22 03:08:26 +05:30
Akash Mahanty
405e9a2a79
waybackpy/save_api.py : Added doc strings and also lint with black.
2022-01-22 00:41:10 +05:30
Akash Mahanty
db551abbf6
lint waybackpy/cdx_api.py and added some doc strings
2022-01-22 00:11:35 +05:30
Akash Mahanty
d13dd4db1a
added notice on waybackpy/wrapper.py that the Url class will cease to exist after 2024-01-01 and also removed unused imports.
2022-01-21 23:14:20 +05:30
Akash Mahanty
d3bb8337a1
make setup.py smarter, now no need to update the URL again and also added more keywords. And in __version__.py updated the __author__
2022-01-21 23:01:09 +05:30
Akash Mahanty
fd5e85420c
waybackpy/availability_api.py : removed unused imports, added doc strings, removed redundant function.
2022-01-21 22:47:44 +05:30
Akash Mahanty
5e9fdb40ce
escape '.' before 'archive.org'
...
escape '.' before 'archive.org' on line 88 so it does not match more hosts than expected.
2022-01-21 19:51:08 +05:30
Akash Mahanty
66e16d6d89
define __repr__ for the Availability API class
2022-01-18 20:34:21 +05:30
Akash Mahanty
16b9bdd7f9
output the file name if known_url and file flag are passed.
2022-01-18 20:14:44 +05:30
Akash Mahanty
7adc01bff2
implement known_urls for cli from the newer interface. Although use of CDX is recommended but backward-compatibility matters.
2022-01-18 20:07:12 +05:30
Akash Mahanty
cc3628ae18
define __str__ for objects of WaybackMachineAvailabilityAPI class, the check for self.JSON ensures that the API was atleast called.
2022-01-16 22:28:12 +05:30
Akash Mahanty
1d751b942b
invoke json, was a bad idea removing it the earlier commit as the end user should not have to call it
2022-01-16 22:15:25 +05:30
Akash Mahanty
261a867a21
near() method of WaybackMachineAvailabilityAPI return self to preserve past behaviour
2022-01-16 21:53:54 +05:30
Akash Mahanty
2e487e88d3
define __len__ on Url objects, if any method not used prior to len op then default to len of oldest archive.
2022-01-16 21:29:43 +05:30
Akash Mahanty
c8d0ad493a
defined __str__ for Url objects, print func should print the url.
2022-01-16 21:22:43 +05:30
Akash Mahanty
4e68cd5743
Create separate module for the 3 different APIs also CDX is now CLI supported.
2022-01-02 14:14:45 +05:30
akamhy
a7b805292d
changes made for v2.4.4 (update download_url) ( #100 )
...
* v2.4.4 (update download_url)
* v2.4.4 (update __version__)
* +1
add jonasjancarik
2021-09-03 11:28:26 +05:30
Jonáš Jančařík
6dc6124dc4
Raise error on a 509 response (too many sessions) ( #99 )
...
* Raise error on a 509 response (too many sessions)
When the response code is 509, raise an error with an explanation (based on the actual error message contained in the response HTML).
* Raise error on a 509 response (too many sessions) - linting
2021-09-03 08:04:36 +05:30
Jens Finkhaeuser
5a7fc7d568
Fix typo ( #95 )
2021-04-13 16:58:34 +05:30
Akash Mahanty
5a9c861cad
v2.4.3 ( #94 )
...
* 2.4.3
* 2.4.3
2021-04-02 10:41:59 +05:30
Akash Mahanty
dd1917c77e
added RedirectSaveError - for failed saves if the URL is a permanent … ( #93 )
...
* added RedirectSaveError - for failed saves if the URL is a permanent redirect.
* check if url is redirect before throwing exceptions, res.url is the redirect url if redirected at all
* update tests and cli errors
2021-04-02 10:38:17 +05:30