Akash Mahanty
44861230a5
fix codeql warnings, nothing to worry about here.
2022-02-17 23:42:43 +05:30
Akash Mahanty
6cd9ae1e09
added tests
2022-02-17 23:28:59 +05:30
Akash Mahanty
b541d8dc0e
added cli support for sort, use-pagination and closest
2022-02-17 21:15:05 +05:30
Akash Mahanty
0e7255c1d8
add attrs use_pagination and closest, which are can be used to use the pagination API and lookup archive close to a timestamp respectively. And now to get out of infinte blank pages loop just check for two succesive black and not total two blank pages while using the CDX server API.
2022-02-17 21:13:42 +05:30
Akash Mahanty
28f4b579f2
create check_for_blocked_site for handling the BlockedSiteError for sites that are blocking wayback machine by their robots.txt policy
2022-02-17 21:07:05 +05:30
Akash Mahanty
faa97e5877
add BlockedSiteError for cases when archiving is blocked by site's robots.txt
2022-02-17 21:04:21 +05:30
Akash Mahanty
b4adf4a372
add sort param support in CDX API class
...
see https://nla.github.io/outbackcdx/api.html#operation/query
sort takes string input which must be one of the follwoing:
- default
- closest
- reverse
This commit shall help in closing issue at https://github.com/akamhy/waybackpy/issues/155
2022-02-17 12:08:48 +05:30
Akash Mahanty
f63c6adf79
Trigger Build
2022-02-09 17:29:19 +05:30
eggplants
b4d3393ef1
fix: move metadata from __init__.py into setup.cfg ( #153 )
2022-02-09 17:20:23 +05:30
Akash Mahanty
cd5c3c61a5
fix imports with isort
3.0.3
2022-02-09 16:18:25 +05:30
Akash Mahanty
87fb5ecd58
remove latest version funcs from utils, they were unused.
2022-02-09 16:12:30 +05:30
Akash Mahanty
5954fcc646
format with black
2022-02-09 15:51:11 +05:30
Akash Mahanty
89016d433c
added trove Typing :: Typed and Development Status :: 5 - Production/Stable
2022-02-09 15:47:38 +05:30
Akash Mahanty
edaa1d5d54
update value to the new limit.
2022-02-09 15:40:38 +05:30
Akash Mahanty
16f94db144
incr version to v3.0.3
2022-02-09 14:33:16 +05:30
Akash Mahanty
25eb709ade
improve doc strings and comments and remove useless exceptions.
2022-02-09 14:32:15 +05:30
Akash Mahanty
6d233f24fc
apply isort
2022-02-09 11:20:59 +05:30
Akash Mahanty
ec341fa8b3
refactor code in cli module
2022-02-09 11:20:10 +05:30
Akash Mahanty
cf18090f90
fix typo
2022-02-09 09:52:20 +05:30
Akash Mahanty
81162eebd0
issues with HN
2022-02-08 21:28:25 +05:30
Akash Mahanty
ca4f79a2e3
+ jfinkhaeuser and rafael ( #150 )
2022-02-08 20:34:34 +05:30
Akash Mahanty
27f2727049
add cli alias for --start-timestamp(--from) and --end-timestamp(--to) to conform with the CDX API docs.
2022-02-08 20:12:19 +05:30
Akash Mahanty
118dc6c523
add test for wrapper module
2022-02-08 20:08:44 +05:30
Akash Mahanty
1216ffbc70
lint and refactor cli module
2022-02-08 20:06:17 +05:30
Akash Mahanty
d58a5f0ee5
explicitly exculde some dirs from flake8 check
2022-02-08 18:59:13 +05:30
Akash Mahanty
7e7412d9d1
remove deepsource, LGTM is better and has fewer False Postives.
2022-02-08 18:49:44 +05:30
Akash Mahanty
70c38c5a60
+ codecov badge
2022-02-08 17:49:05 +05:30
Akash Mahanty
f8bf9c16f9
Add tests ( #149 )
...
* enable codecov
* fix save_urls_on_file
* increase the limit of CDX to 25000 from 5000. 5X increase.
* added test for the CLI module
* make flake 8 happy
* make mypy happy
2022-02-08 17:46:59 +05:30
Akash Mahanty
2bbfee7b2f
replace non-ASCII emojis with GitHub hosted equivalent images ( #148 )
2022-02-08 11:43:32 +05:30
deepsource-autofix[bot]
7317bd7183
Remove blank lines after docstring ( #146 )
...
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-02-08 10:14:20 +05:30
deepsource-autofix[bot]
e0dfbe0b7d
Fix comparison constant position ( #145 )
...
* Fix comparison constant position
* format with black
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Akash Mahanty <akamhy@yahoo.com >
2022-02-08 10:06:23 +05:30
eggplants
0b631592ea
Improve pylint score ( #142 )
...
* fix: errors to improve pylint scores
* fix: test
* fix
* add: flake ignore rule to pip8speaks conf
* fix
* add: test patterns to deepsource conf
2022-02-08 06:42:20 +09:00
Akash Mahanty
d3a8f343f8
+ [eggplants]( https://github.com/eggplants ) ( #143 )
2022-02-08 01:41:10 +05:30
Akash Mahanty
97f8b96411
added docstrings, added some static type hints and also lint. ( #141 )
...
* added docstrings, added some static type hints and also lint.
* added doc strings and changed some internal variable names for more clarity.
* make flake8 happy
* add descriptive docstrings and type hints in waybackpy/cdx_snapshot.py
* remove useless code and add docstrings and also lint using pylint.
* remove unwarented test
* added docstrings, lint using pylint and add a raise on 509 SC
* added docstrings and lint with pylint
* lint
* add doc strings and lint
* add docstrings and lint
2022-02-07 19:40:37 +05:30
DeepSource Bot
004ff26196
Add .deepsource.toml
2022-02-07 12:55:57 +00:00
Akash Mahanty
a772c22431
explicitly tell pep8speaks that mll is 88.
2022-02-06 21:00:15 +05:30
Akash Mahanty
b79f1c471e
Merge pull request #135 from eggplants/fix_cli
...
Fix cli.py
2022-02-05 16:54:36 +05:30
Akash Mahanty
f49d67a411
Merge pull request #136 from eggplants/429_error
...
Add TooManyRequestsError
2022-02-05 11:28:27 +05:30
Akash Mahanty
ad8bd25633
added badge of codacy ( #139 )
2022-02-05 10:05:17 +05:30
eggplants
d2a3946425
fix: escape banner
2022-02-05 10:12:27 +09:00
eggplants
7b6401d59b
fix: delete useless conds
2022-02-05 06:20:03 +09:00
eggplants
ed6160c54f
add: TooManyRequestsError
2022-02-05 06:19:02 +09:00
eggplants
fcab19a40a
fix: cli
...
print error message to stderr and specify defaults of url
2022-02-05 05:55:04 +09:00
eggplants
5f3cd28046
Fix Pylint errors were pointed out by codacy ( #133 )
...
* fix: pylint errors were pointed out by codacy
* fix: line length
* fix: help text
* fix: revert
https://stackoverflow.com/a/64477857 makes cli unusable
* fix: cli error and refactor codes
2022-02-05 05:25:40 +09:00
Akash Mahanty
9d9cc3328b
add .pep8speaks.yml, override deafult
2022-02-05 00:53:38 +05:30
Akash Mahanty
b69e4dff37
rename params of main in cli.py to avoid using built-ins ( #132 )
...
* rename params of main in cli.py to avoid using built-ins
* Fix Line 32:80: E501 line too long (102 > 79 characters)
2022-02-05 00:30:35 +05:30
eggplants
d8cabdfdb5
Typing ( #128 )
...
* fix: CI yml name
* add: mypy configuraion
* add: type annotation to waybackpy modules
* add: type annotation to test modules
* fix: mypy command
* add: types-requests to dev deps
* fix: disable max-line-length
* fix: move pytest.ini into setup.cfg
* add: urllib3 to deps
* fix: Retry (ref: https://github.com/python/typeshed/issues/6893 )
* fix: f-string
* fix: shorten long lines
* add: staticmethod decorator to no-self-use methods
* fix: str(headers)->headers_str
* fix: error message
* fix: revert "str(headers)->headers_str" and ignore assignment CaseInsensitiveDict with str
* fix: mypy error
2022-02-05 03:23:36 +09:00
eggplants
320ef30371
fix: format md and yml ( #129 )
2022-02-04 22:31:46 +05:30
eggplants
e61447effd
Format and lint codes and fix packaging ( #125 )
...
* add: configure files (setup.py->setup.py+setup.cfg+pyproject.toml)
* add: __download_url__
* format with black and isort
* fix: flake8 section in setup.cfg
* add: E501 to flake ignore
* fix: metadata.name does not accept attr
* fix: merge __version__.py into __init__.py
* fix: flake8 errors in tests/
* fix: datetime.datetime -> datetime
* fix: banner
* fix: ignore W605 for banner
* fix: way to install deps in CI
* add: versem to setuptools
* fix: drop python<=3.6 (#126 ) from package and CI
2022-02-03 19:13:39 +05:30
Akash Mahanty
947647f2e7
Merge pull request #124 from eggplants/fix_save_retry
...
Fix save retry mechanism
2022-02-03 18:01:51 +05:30