From d94d12dbc5fb3e3712b2f1aea949013cb8213e28 Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Thu, 26 Aug 2021 10:20:53 -0700 Subject: [PATCH 1/6] add foo parser template info --- CONTRIBUTING.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1fba610f..6ddf3e23 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,11 +15,12 @@ Pull requests are the best way to propose changes to the codebase (we use [Githu 1. Open an issue to discuss the new feature, bug fix, or parser before opening a pull request. For new parsers, it is important to agree upon a schema before developing the parser. 2. Fork the repo and create your branch from `dev`, if available, otherwise `master`. -3. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests. -4. Documentation is auto-generated from docstrings, so ensure they are clear and accurate. -5. Ensure the test suite passes. (Note: "**America/Los_Angeles**" timezone should be configured on the test system) -6. Make sure your code lints. -7. Issue that pull request! +3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. +4. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests. +5. Documentation is auto-generated from docstrings, so ensure they are clear and accurate. +6. Ensure the test suite passes. (Note: "**America/Los_Angeles**" timezone should be configured on the test system) +7. Make sure your code lints. +8. Issue that pull request! ## Parser Schema Guidelines - Try to keep the schema as flat as possible - typically a list of flat dictionaries From f857b7fbf7862ddd7eb5ab20470f5dbaacc62119 Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Thu, 26 Aug 2021 10:25:37 -0700 Subject: [PATCH 2/6] add custom parser folder info --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6ddf3e23..e61b5b3f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,7 @@ Pull requests are the best way to propose changes to the codebase (we use [Githu 1. Open an issue to discuss the new feature, bug fix, or parser before opening a pull request. For new parsers, it is important to agree upon a schema before developing the parser. 2. Fork the repo and create your branch from `dev`, if available, otherwise `master`. -3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. +3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. You can even place a new parser python module file in the [parser plugin directory](https://github.com/kellyjonbrazil/jc#custom-parsers) to get started right away with a standard `jc` installation. 4. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests. 5. Documentation is auto-generated from docstrings, so ensure they are clear and accurate. 6. Ensure the test suite passes. (Note: "**America/Los_Angeles**" timezone should be configured on the test system) From 7661e7f27a4e5c379ea37c7f95496aca2c1e2910 Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Thu, 26 Aug 2021 10:27:11 -0700 Subject: [PATCH 3/6] formatting --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e61b5b3f..872fc240 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,7 @@ Pull requests are the best way to propose changes to the codebase (we use [Githu 1. Open an issue to discuss the new feature, bug fix, or parser before opening a pull request. For new parsers, it is important to agree upon a schema before developing the parser. 2. Fork the repo and create your branch from `dev`, if available, otherwise `master`. -3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. You can even place a new parser python module file in the [parser plugin directory](https://github.com/kellyjonbrazil/jc#custom-parsers) to get started right away with a standard `jc` installation. +3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. You can even place a new parser python module file in the [parser plugin directory](https://github.com/kellyjonbrazil/jc#custom-parsers) to get started right away with just a standard `jc` installation. 4. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests. 5. Documentation is auto-generated from docstrings, so ensure they are clear and accurate. 6. Ensure the test suite passes. (Note: "**America/Los_Angeles**" timezone should be configured on the test system) From 1a9fd2139d156a51bf2d25db951e47e96da77f7d Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Thu, 26 Aug 2021 10:29:13 -0700 Subject: [PATCH 4/6] add "For new parsers:" --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 872fc240..a76e1b5d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -15,7 +15,7 @@ Pull requests are the best way to propose changes to the codebase (we use [Githu 1. Open an issue to discuss the new feature, bug fix, or parser before opening a pull request. For new parsers, it is important to agree upon a schema before developing the parser. 2. Fork the repo and create your branch from `dev`, if available, otherwise `master`. -3. Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. You can even place a new parser python module file in the [parser plugin directory](https://github.com/kellyjonbrazil/jc#custom-parsers) to get started right away with just a standard `jc` installation. +3. For new parsers: Use the `jc/parsers/foo.py` parser as a [template](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) to get started. You can even place a new parser python module file in the [parser plugin directory](https://github.com/kellyjonbrazil/jc#custom-parsers) to get started right away with just a standard `jc` installation. 4. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests. 5. Documentation is auto-generated from docstrings, so ensure they are clear and accurate. 6. Ensure the test suite passes. (Note: "**America/Los_Angeles**" timezone should be configured on the test system) From 9c1ad92fed9a4b3a0da66d6b71deec8d608edfa9 Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Thu, 26 Aug 2021 10:31:51 -0700 Subject: [PATCH 5/6] fix examples --- CONTRIBUTING.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a76e1b5d..0d397589 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -35,11 +35,11 @@ Bad: ``` { "Interface 1": [ - 192.168.1.1, - 172.16.1.1 + "192.168.1.1", + "172.16.1.1" ], "Wifi Interface 1": [ - 10.1.1.1 + "10.1.1.1" ] } ``` @@ -49,14 +49,14 @@ Good: { "interface": "Interface 1", "ip_addresses": [ - 192.168.1.1, - 172.16.1.1 + "192.168.1.1", + "172.16.1.1" ] }, { "interface": "Wifi Interface 1", "ip_addresses": [ - 10.1.1.1 + "10.1.1.1" ] } ] From 528aac7ad87feae2093aba563c83a29ebf75e5f9 Mon Sep 17 00:00:00 2001 From: Kelly Brazil Date: Wed, 8 Sep 2021 07:00:50 -0700 Subject: [PATCH 6/6] use subprocess in python example --- README.md | 23 ++++------------------- templates/readme_template | 23 ++++------------------- 2 files changed, 8 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index 2d959ea6..0344e460 100644 --- a/README.md +++ b/README.md @@ -31,28 +31,13 @@ $ jc dig example.com | jq -r '.[].answer[].data' ``` The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary, or list of dictionaries, instead of JSON: ```python +>>> import subprocess >>> import jc.parsers.dig >>> ->>> data = '''; <<>> DiG 9.10.6 <<>> example.com -... ;; global options: +cmd -... ;; Got answer: -... ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64612 -... ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 -... -... ;; OPT PSEUDOSECTION: -... ; EDNS: version: 0, flags:; udp: 4096 -... ;; QUESTION SECTION: -... ;example.com. IN A -... -... ;; ANSWER SECTION: -... example.com. 29658 IN A 93.184.216.34 -... -... ;; Query time: 52 msec -... ;; SERVER: 2600:1700:bab0:d40::1#53(2600:1700:bab0:d40::1) -... ;; WHEN: Fri Apr 16 16:13:00 PDT 2021 -... ;; MSG SIZE rcvd: 56''' +>>> cmd_output = subprocess.check_output(['dig', 'example.com'], text=True) +>>> data = jc.parsers.dig.parse(cmd_output) >>> ->>> jc.parsers.dig.parse(data) +>>> data [{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', 'flags': ['qr', 'rd', 'ra'], 'query_num': 1, 'answer_num': 1, 'authority_num': 0, 'additional_num': 1, 'opt_pseudosection': {'edns': {'version': 0, 'flags': [], 'udp': 4096}}, 'question': {'name': 'example.com.', 'class': 'IN', 'type': 'A'}, 'answer': [{'name': 'example.com.', diff --git a/templates/readme_template b/templates/readme_template index da09d75d..8466913a 100644 --- a/templates/readme_template +++ b/templates/readme_template @@ -31,28 +31,13 @@ $ jc dig example.com | jq -r '.[].answer[].data' ``` The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary, or list of dictionaries, instead of JSON: ```python +>>> import subprocess >>> import jc.parsers.dig >>> ->>> data = '''; <<>> DiG 9.10.6 <<>> example.com -... ;; global options: +cmd -... ;; Got answer: -... ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64612 -... ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 -... -... ;; OPT PSEUDOSECTION: -... ; EDNS: version: 0, flags:; udp: 4096 -... ;; QUESTION SECTION: -... ;example.com. IN A -... -... ;; ANSWER SECTION: -... example.com. 29658 IN A 93.184.216.34 -... -... ;; Query time: 52 msec -... ;; SERVER: 2600:1700:bab0:d40::1#53(2600:1700:bab0:d40::1) -... ;; WHEN: Fri Apr 16 16:13:00 PDT 2021 -... ;; MSG SIZE rcvd: 56''' +>>> cmd_output = subprocess.check_output(['dig', 'example.com'], text=True) +>>> data = jc.parsers.dig.parse(cmd_output) >>> ->>> jc.parsers.dig.parse(data) +>>> data [{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', 'flags': ['qr', 'rd', 'ra'], 'query_num': 1, 'answer_num': 1, 'authority_num': 0, 'additional_num': 1, 'opt_pseudosection': {'edns': {'version': 0, 'flags': [], 'udp': 4096}}, 'question': {'name': 'example.com.', 'class': 'IN', 'type': 'A'}, 'answer': [{'name': 'example.com.',