JSON Manipulation
There are many ways to manipulate JSON. I reviewed a few rapid ways today, which are using a command line tool called jq and libraries that support JSONPath query language. jq Awk is a powerful domain-specific language for text processing, but it lacks built-in support for manipulating hierarchical data structures like trees. jq fills this gap by providing a tool for transforming JSON data. However, one potential drawback is that jq requires a separate installation, which may add complexity to my workflow. So I write a shell script using jq to extract user's names and user's urls from a cURL response, and then output it in TSV format. curl -s 'https://mstdn.in.th/api/v1/timelines/public?limit=10' | jq '.[].account | [.username, .url] | @tsv' -r The result looks like this: kuketzblog https://social.tchncs.de/@kuketzblog cats https://social.goose.rodeo/@cats AlJazeera https://flipboard.com/@AlJazeera TheHindu https://flipboard.com/@TheHindu GossiTheDog https://cyberplace.social/@GossiTheDog kuketzblog https://social.tchncs.de/@kuketzblog weeklyOSM https://en.osm.town/@weeklyOSM juanbellas https://masto.es/@juanbellas noborusudou https://misskey.io/@noborusudou jerryd https://mastodon.social/@jerryd With a TSV file, we can use Awk, sed, etc. to manipulate them as usual. JSONPath JSONPath, which was explained in RFC 9535, is supported by many libraries and applications, e.g. PostgreSQL. Still, I try to it in Python by the jsonpath_nq library. from jsonpath_ng import parse import requests res = requests.get("https://mstdn.in.th/api/v1/timelines/public?limit=10") compiled_path = parse("$[*].account") for matched_node in compiled_path.find(res.json()): print(matched_node.value["username"] + "\t" + matched_node.value["url"]) It gave the same result to the shell script above. The code is a bit longer than the shell script. However, it can be integrated with many Python libraries.
There are many ways to manipulate JSON. I reviewed a few rapid ways today, which are using a command line tool called jq and libraries that support JSONPath query language.
jq
Awk is a powerful domain-specific language for text processing, but it lacks built-in support for manipulating hierarchical data structures like trees. jq fills this gap by providing a tool for transforming JSON data. However, one potential drawback is that jq requires a separate installation, which may add complexity to my workflow.
So I write a shell script using jq to extract user's names and user's urls from a cURL response, and then output it in TSV format.
curl -s 'https://mstdn.in.th/api/v1/timelines/public?limit=10' | jq '.[].account | [.username, .url] | @tsv' -r
The result looks like this:
kuketzblog https://social.tchncs.de/@kuketzblog
cats https://social.goose.rodeo/@cats
AlJazeera https://flipboard.com/@AlJazeera
TheHindu https://flipboard.com/@TheHindu
GossiTheDog https://cyberplace.social/@GossiTheDog
kuketzblog https://social.tchncs.de/@kuketzblog
weeklyOSM https://en.osm.town/@weeklyOSM
juanbellas https://masto.es/@juanbellas
noborusudou https://misskey.io/@noborusudou
jerryd https://mastodon.social/@jerryd
With a TSV file, we can use Awk, sed, etc. to manipulate them as usual.
JSONPath
JSONPath, which was explained in RFC 9535, is supported by many libraries and applications, e.g. PostgreSQL. Still, I try to it in Python by the jsonpath_nq library.
from jsonpath_ng import parse
import requests
res = requests.get("https://mstdn.in.th/api/v1/timelines/public?limit=10")
compiled_path = parse("$[*].account")
for matched_node in compiled_path.find(res.json()):
print(matched_node.value["username"] + "\t" + matched_node.value["url"])
It gave the same result to the shell script above. The code is a bit longer than the shell script. However, it can be integrated with many Python libraries.
