PHP Classes
elePHPant
Icontem

really useful with big files (250 mb).

Recommend this page to a friend!

      Zip  >  All threads  >  really useful with big files (250 mb).  >  (Un) Subscribe thread alerts  
Subject:really useful with big files (250 mb).
Summary:Package rating comment
Messages:14
Author:Francisco Velazquez
Date:2010-09-06 14:57:58
Update:2010-11-13 18:14:28
 

 

Francisco Velazquez rated this package as follows:

Utility: Good
Consistency: Sufficient
Examples: Sufficient
  1 - 10   11 - 14  

  1. really useful with big files (250 mb).   Reply   Report abuse  
Francisco Velazquez - 2010-09-06 14:57:58
really useful with big files (250 mb).

  2. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-06 16:51:14 - In reply to message 1 from Francisco Velazquez
Yikes, 250MB? I didn't even know it could do that :)

Is it one 250MB file zipped, or that the resulting archive was 250MB?

  3. Re: really useful with big files (250 mb).   Reply   Report abuse  
Francisco Velazquez - 2010-09-07 13:54:06 - In reply to message 2 from Asbjorn Grandt
a 250 MB CSV File the result ended up in 12.5 Mb, however now im trying to zip a 490MB MB CSV and im running out of memory (512MB), any thoughts?

  4. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-07 14:11:23 - In reply to message 3 from Francisco Velazquez
I think it's a limit in the PHP server. I'll have a look at a possible solution.

Is the CSV generated or from a file?

  5. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-07 14:27:50 - In reply to message 3 from Francisco Velazquez
Have you tried using ini_set with Memory limit?
ini_set("memory_limit","768M");

I have no idea if it'll work tough.

  6. Re: really useful with big files (250 mb).   Reply   Report abuse  
Francisco Velazquez - 2010-09-07 18:33:15 - In reply to message 5 from Asbjorn Grandt
Nope I keep exhausting the memory, i believe that the hole file its being loaded into memory so it can later perform the compression, is there a way to store in a temp file pieces of the compressed file and once it all its compressed put it together into a file (zip).

  7. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-07 19:06:55 - In reply to message 6 from Francisco Velazquez
I'm working on a better solution that'll allow very large files...I hope.

  8. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-07 20:00:56 - In reply to message 7 from Asbjorn Grandt
I have "something". But there are some bugs to iron out. I did (accidentally) manage to stuff well over 2GB into a zip file, and it didn't flinch, however there seem to be some undocumented aspects regarding the gzwrite and zlib implementation in php that are giving me the fits.
Above a certain size, it start ...falling apart, as if it's not meant to go above 2MB files or something like that. It certainly changes it's behaviour at around that mark.

I'll keep looking.

  9. Re: really useful with big files (250 mb).   Reply   Report abuse  
Francisco Velazquez - 2010-09-07 22:07:36 - In reply to message 8 from Asbjorn Grandt
thanks man, meanwhile ill keep looking.

  10. Re: really useful with big files (250 mb).   Reply   Report abuse  
Asbjorn Grandt - 2010-09-08 19:44:24 - In reply to message 9 from Francisco Velazquez
Try the Zip_2 I just uploaded. The code is a mess and need a severe clean up, however it should allow you to create larger files.

I'll be adding the option to add a file directly, but till then you have to use something like this:

$zip->openStream("filename in zip");

$fh = fopen("filename", "rb");
$filestat = fstat($fh);
$length = $filestat['size'];
$pos = 0;
$readStep = 16384;
while (($pos+$readStep) < $length) {
$zip->addStreamData(fread($fh , $readStep));
$pos += $readStep;
}
if ($pos < $length) {
$zip->addStreamData(read($this->zipFile, $length-$pos));
}

$zip->closeStream();

 
  1 - 10   11 - 14