0
Python training array too large

0So I have over 30 million objects that I need to use as my training data. My Issue is simple: When I create my training array by an iterative appending process, at a certain threshold, the list becomes too large and python gets killed. What is a way to get around this? I have been trying to figure this out for hours and keep coming up short!

Code example for creating training array

training_array = []
for ...:
    data = #load data from somewhere
    data_array = [x for x in data] #some large array, 2-3 million objects  
    for item in data_array:
        training_array.append(item.a + item.b)


Python 02-05-22, 3:07 p.m. jagath
0
Great things you’ve always shared with us. Just keep writing this kind of posts.The time which was wasted in traveling for tuition now it can be used for studies.Thanks Ultimate Home Improvement
25-05-22, 3:53 p.m. browning


0
I love the way you write and share your niche! Very interesting and different! Keep it coming! Ethereum NFT
02-06-22, 6:26 p.m. browning


0
Great job for publishing such a beneficial web site. Your web log isn’t only useful but it is additionally really creative too. trash chute door handle
09-06-22, 1:41 p.m. Natalie0


0
You delivered such an impressive piece to read, giving every subject enlightenment for us to gain information. Thanks for sharing such information with us due to which my several concepts have been cleared. Buy Whisky online free delivery
20-06-22, 12:56 p.m. browning


0
Wonderful article. Fascinating to read. I love to read such an excellent article. Thanks! It has made my task more and extra easy. Keep rocking. Buy Whisky online cheap
20-06-22, 12:56 p.m. browning


0
Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! dinajpur eye hospital doctor list
20-06-22, 3:32 p.m. browning


0
Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Sale
25-06-22, 5:35 p.m. browning


0
ehternet cables are still the ones that i use for my home networking applications.google posts
Oops! Looks like spam! Waiting for moderator approval.
30-08-22, 2:22 p.m. STEPHEN007

Login to add comment
Oops! Looks like spam! Waiting for moderator approval


0
Hello there, May I download that snapshot and make use of it on my personal weblog?linkedin content strategy
Oops! Looks like spam! Waiting for moderator approval.
30-08-22, 4:14 p.m. STEPHEN007

Login to add comment
Oops! Looks like spam! Waiting for moderator approval


Log-in to answer to this question.