How to join two json files in python instead of nesting for loop

General Tech Bugs & Fixes 2 years ago

0 2 0 0 0 tuteeHUB earn credit +10 pts

5 Star Rating 1 Rating

Posted on 16 Aug 2022, this text provides information on Bugs & Fixes related to General Tech. Please note that while accuracy is prioritized, the data presented might not be entirely correct or up-to-date. This information is offered for general knowledge and informational purposes only, and should not be considered as a substitute for professional advice.

Take Quiz To Earn Credits!

Turn Your Knowledge into Earnings.

tuteehub_quiz

Answers (2)

Post Answer
profilepic.png
manpreet Tuteehub forum best answer Best Answer 2 years ago

 

Every time i will get 500 records form file1 to join file2 which contains more than 100000 records it costs two minutes !!

with open(file1,'r') as f1,open(file2,'r') as f2:
    a=json.load(f1)
    b=json.load(f2)
    list_a=[]
    for i in range(len(a)):
        for n in range(len(b)):
            if b[n]["id"]==a[i]["id"]:
                list_a.append(dict(b[n],**a[i]))
with open(result,'w') as f3:
    json.dump(list_a, f3,sort_keys=True, ensure_ascii=False)

File1:

[{ "id":"1", "name":"Tom" }, 
{ "id":"2", "name":"Jim" }, 
{ "id":"3", "name":"Bob" }, 
{ "id":"4", "name":"Jeny" },  
{ "id":"5", "name":"Lara" }, 
{ "id":"6", "name":"Lin" }, 
{ "id":"7", "name":"Kim" }, 
{ "id":"8", "name":"Jack" }, 
{ "id":"9", "name":"Tony" }]

File 2:

[ { "id":"1", "Details":[ { "label":"jcc", "hooby":"Swimming" }, { "label":"hkt", "hooby":"Basketball" }, ] }, 
{ "id":"2", "Details":[ { 
                                                
                                                
0 views
0 shares
profilepic.png
manpreet 2 years ago

 

I don't have the experience to know if this would speed it up. The solution below provided by Eugene Yarmash seems more reliable. I also don't have the big files to test speed, but you can try and see if using collections would speed up the iteration. I'd actually be curious myself if it would change anything:

File1 = [ { "id":"1", "name":"Tom" }, { "id":"2", "name":"Jim" }, { "id":"3", "name":"Bob" }, { "id":"4", "name":"Jeny" }, { "id":"5", "name":"Lara" }, { "id":"6", "name":"Lin" }, { "id":"7", "name":"Kim" }, { "id":"8", "name":"Jack" }, { "id":"9", "name":"Tony" } ]
File2 = [ { "id":"1", "Details":[ { "label":"jcc", "hooby":"Swimming" }, { "label":"hkt", "hooby":"Basketball" }, ] }, { "id":"2", "Details":[ { "label":"NTC", "hooby":"Games" } ] } ] 

from collections import defaultdict

d = defaultdict(dict)
for l in (File1, File2):
    for elem in l:
        d[elem['id']].update(elem)
Result = dict(d)

0 views   0 shares

No matter what stage you're at in your education or career, TuteeHub will help you reach the next level that you're aiming for. Simply,Choose a subject/topic and get started in self-paced practice sessions to improve your knowledge and scores.