Commit Graph

491 Commits

Author SHA1 Message Date
Akash Mahanty
7930cacedf remove the license section from readme
This does not mean that I'm waving the copyrights rather just formatting the README
2022-03-15 20:03:44 +05:30
Akash Mahanty
cfd977135d Update CITATION.cff (#169) 2022-03-04 11:48:49 +05:30
eggplants
7a5e0bfdaf fix: cff format (#168) 2022-03-04 03:10:27 +05:30
eggplants
48dcda8020 add: typed marker (PEP561) (#167) 2022-03-03 19:05:43 +05:30
eggplants
3ed2170a32 add: CITATION.cff (#166) 2022-03-03 19:05:23 +05:30
Akash Mahanty
d6ef55020c undo drop python3.6, see #162 (#163) 3.0.5 2022-02-18 21:54:33 +05:30
Akash Mahanty
2650943f9d v3.0.4 (#160)
* Update README.md

* Update README.md

* update asciinema link

* v3.0.4

* update video link
3.0.4
2022-02-18 16:05:58 +05:30
Akash Mahanty
4b218d35cb Cdx based oldest newest and near (#159)
* implement oldest newest and near methods in the cdx interface class, now cli uses the cdx methods instead of availablity api methods.

* handle the closest parameter derivative methods more efficiently and also handle exceptions gracefully.

* update test code
2022-02-18 13:17:40 +05:30
Akash Mahanty
f990b93f8a Add sort, use_pagination and closest (#158)
* add sort param support in CDX API class

see https://nla.github.io/outbackcdx/api.html#operation/query

sort takes string input which must be one of the follwoing:
- default
- closest
- reverse

This commit shall help in closing issue at https://github.com/akamhy/waybackpy/issues/155

* add BlockedSiteError for cases when archiving is blocked by site's robots.txt

* create check_for_blocked_site for handling the BlockedSiteError for sites that are blocking wayback machine by their robots.txt policy

* add attrs use_pagination and closest, which are can be used to use the pagination API and lookup archive close to a timestamp respectively. And now to get out of infinte blank pages loop just check for two succesive black and not total two blank pages while using the CDX server API.

* added cli support for sort, use-pagination and closest

* added tests

* fix codeql warnings, nothing to worry about here.

* fix save test for archive_url
2022-02-18 00:24:14 +05:30
Akash Mahanty
3a44a710d3 add sort param support in CDX API class (#156)
see https://nla.github.io/outbackcdx/api.html#operation/query

sort takes string input which must be one of the follwoing:
- default
- closest
- reverse

This commit shall help in closing issue at https://github.com/akamhy/waybackpy/issues/155
2022-02-17 12:17:23 +05:30
Akash Mahanty
f63c6adf79 Trigger Build 2022-02-09 17:29:19 +05:30
eggplants
b4d3393ef1 fix: move metadata from __init__.py into setup.cfg (#153) 2022-02-09 17:20:23 +05:30
Akash Mahanty
cd5c3c61a5 fix imports with isort 3.0.3 2022-02-09 16:18:25 +05:30
Akash Mahanty
87fb5ecd58 remove latest version funcs from utils, they were unused. 2022-02-09 16:12:30 +05:30
Akash Mahanty
5954fcc646 format with black 2022-02-09 15:51:11 +05:30
Akash Mahanty
89016d433c added trove Typing :: Typed and Development Status :: 5 - Production/Stable 2022-02-09 15:47:38 +05:30
Akash Mahanty
edaa1d5d54 update value to the new limit. 2022-02-09 15:40:38 +05:30
Akash Mahanty
16f94db144 incr version to v3.0.3 2022-02-09 14:33:16 +05:30
Akash Mahanty
25eb709ade improve doc strings and comments and remove useless exceptions. 2022-02-09 14:32:15 +05:30
Akash Mahanty
6d233f24fc apply isort 2022-02-09 11:20:59 +05:30
Akash Mahanty
ec341fa8b3 refactor code in cli module 2022-02-09 11:20:10 +05:30
Akash Mahanty
cf18090f90 fix typo 2022-02-09 09:52:20 +05:30
Akash Mahanty
81162eebd0 issues with HN 2022-02-08 21:28:25 +05:30
Akash Mahanty
ca4f79a2e3 + jfinkhaeuser and rafael (#150) 2022-02-08 20:34:34 +05:30
Akash Mahanty
27f2727049 add cli alias for --start-timestamp(--from) and --end-timestamp(--to) to conform with the CDX API docs. 2022-02-08 20:12:19 +05:30
Akash Mahanty
118dc6c523 add test for wrapper module 2022-02-08 20:08:44 +05:30
Akash Mahanty
1216ffbc70 lint and refactor cli module 2022-02-08 20:06:17 +05:30
Akash Mahanty
d58a5f0ee5 explicitly exculde some dirs from flake8 check 2022-02-08 18:59:13 +05:30
Akash Mahanty
7e7412d9d1 remove deepsource, LGTM is better and has fewer False Postives. 2022-02-08 18:49:44 +05:30
Akash Mahanty
70c38c5a60 + codecov badge 2022-02-08 17:49:05 +05:30
Akash Mahanty
f8bf9c16f9 Add tests (#149)
* enable codecov

* fix save_urls_on_file

* increase the limit of CDX to 25000 from 5000. 5X increase.

* added test for the CLI module

* make flake 8 happy

* make mypy happy
2022-02-08 17:46:59 +05:30
Akash Mahanty
2bbfee7b2f replace non-ASCII emojis with GitHub hosted equivalent images (#148) 2022-02-08 11:43:32 +05:30
deepsource-autofix[bot]
7317bd7183 Remove blank lines after docstring (#146)
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2022-02-08 10:14:20 +05:30
deepsource-autofix[bot]
e0dfbe0b7d Fix comparison constant position (#145)
* Fix comparison constant position

* format with black

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Akash Mahanty <akamhy@yahoo.com>
2022-02-08 10:06:23 +05:30
eggplants
0b631592ea Improve pylint score (#142)
* fix: errors to improve pylint scores

* fix: test

* fix

* add: flake ignore rule to pip8speaks conf

* fix

* add: test patterns to deepsource conf
2022-02-08 06:42:20 +09:00
Akash Mahanty
d3a8f343f8 + [eggplants](https://github.com/eggplants) (#143) 2022-02-08 01:41:10 +05:30
Akash Mahanty
97f8b96411 added docstrings, added some static type hints and also lint. (#141)
* added docstrings, added some static type hints and also lint.

* added doc strings and changed some internal variable names for more clarity.

* make flake8 happy

* add descriptive docstrings and type hints in waybackpy/cdx_snapshot.py

* remove useless code and add docstrings and also lint using pylint.

* remove unwarented test

* added docstrings, lint using pylint and add a raise on 509 SC

* added docstrings and lint with pylint

* lint

* add doc strings and lint

* add docstrings and lint
2022-02-07 19:40:37 +05:30
DeepSource Bot
004ff26196 Add .deepsource.toml 2022-02-07 12:55:57 +00:00
Akash Mahanty
a772c22431 explicitly tell pep8speaks that mll is 88. 2022-02-06 21:00:15 +05:30
Akash Mahanty
b79f1c471e Merge pull request #135 from eggplants/fix_cli
Fix cli.py
2022-02-05 16:54:36 +05:30
Akash Mahanty
f49d67a411 Merge pull request #136 from eggplants/429_error
Add TooManyRequestsError
2022-02-05 11:28:27 +05:30
Akash Mahanty
ad8bd25633 added badge of codacy (#139) 2022-02-05 10:05:17 +05:30
eggplants
d2a3946425 fix: escape banner 2022-02-05 10:12:27 +09:00
eggplants
7b6401d59b fix: delete useless conds 2022-02-05 06:20:03 +09:00
eggplants
ed6160c54f add: TooManyRequestsError 2022-02-05 06:19:02 +09:00
eggplants
fcab19a40a fix: cli
print error message to stderr and specify defaults of url
2022-02-05 05:55:04 +09:00
eggplants
5f3cd28046 Fix Pylint errors were pointed out by codacy (#133)
* fix: pylint errors were pointed out by codacy

* fix: line length

* fix: help text

* fix: revert

https://stackoverflow.com/a/64477857 makes cli unusable

* fix: cli error and refactor codes
2022-02-05 05:25:40 +09:00
Akash Mahanty
9d9cc3328b add .pep8speaks.yml, override deafult 2022-02-05 00:53:38 +05:30
Akash Mahanty
b69e4dff37 rename params of main in cli.py to avoid using built-ins (#132)
* rename params of main in cli.py to avoid using built-ins

* Fix Line 32:80: E501 line too long (102 > 79 characters)
2022-02-05 00:30:35 +05:30
eggplants
d8cabdfdb5 Typing (#128)
* fix: CI yml name

* add: mypy configuraion

* add: type annotation to waybackpy modules

* add: type annotation to test modules

* fix: mypy command

* add: types-requests to dev deps

* fix: disable max-line-length

* fix: move pytest.ini into setup.cfg

* add: urllib3 to deps

* fix: Retry (ref: https://github.com/python/typeshed/issues/6893)

* fix: f-string

* fix: shorten long lines

* add: staticmethod decorator to no-self-use methods

* fix: str(headers)->headers_str

* fix: error message

* fix: revert "str(headers)->headers_str" and ignore assignment CaseInsensitiveDict with str

* fix: mypy error
2022-02-05 03:23:36 +09:00