CryptDecode with modifier CRYPT_ARCH_ZIP - How to use? - page 10
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Vasiliy!
Didn't wait for a reply from you. (don't know if MQ had time to implement changes in build 1100)
Decoder:
//+------------------------------------------------------------------+ //| Zip_decoder.mq5 | //| Copyright 2015, Mikalas | //| https://www.mql5.com | //+------------------------------------------------------------------+ // #define uint_size 4 #define hdr_size 2 #define descr_size 12 // struct ZIP_HEADER { uint sign; ushort a_version; ushort bit_flag; ushort comp_method; ushort last_mod_time; ushort last_mod_date; uint crc_32; uint pack_size; uint unpack_size; ushort fn_len; ushort extr_field_len; }; //--- struct ZIP_ENTRYES { uint file_count; string zip_fnames[]; ulong zip_offsets[]; uint pack_size[]; uint unpack_size[]; }; struct DATA_DESCRIPTOR { uint crc_32; uint pack_size; uint unpack_size; }; // DATA_DESCRIPTOR data_descriptor; ZIP_HEADER zip_header; ZIP_ENTRYES zip_entryes; uchar key[] = {1,0,0,0}; bool f_found; uchar f_name[]; bool is_descr; //+------------------------------------------------------------------+ //| Get entryes in ZIP file | //+------------------------------------------------------------------+ uint GetEntryesZip( int handle, ZIP_ENTRYES &f_entr ) { //--- Check valid file handle if ( handle != INVALID_HANDLE ) { f_entr.file_count = 0; //--- ulong file_size = FileSize( handle ); if ( file_size > 0 ) { //---Set file position to "0" FileSeek( handle, 0, SEEK_SET ); //---Build table while ( FileTell( handle ) < ( file_size - uint_size ) ) { zip_header.sign = FileReadInteger( handle, uint_size ); //--- if ( zip_header.sign == 0x04034b50 ) { FileSeek( handle, -4, SEEK_CUR ); f_entr.file_count++; FileReadStruct( handle, zip_header ); //---Check needed version if ( ( zip_header.a_version != 0x0014 ) && ( f_entr.file_count == 1 ) ) { Print( "Unsupported version!" ); f_entr.file_count = 0; return( f_entr.file_count ); } if ( ( zip_header.extr_field_len != 0 ) && ( f_entr.file_count == 1 ) ) { Print( "File compressed using ZIP64 format!" ); f_entr.file_count = 0; return( f_entr.file_count ); } //---Resize out data ArrayResize( f_entr.pack_size, f_entr.file_count ); ArrayResize( f_entr.unpack_size, f_entr.file_count ); ArrayResize( f_entr.zip_fnames, f_entr.file_count ); ArrayResize( f_entr.zip_offsets, f_entr.file_count ); //---Check for sizes using extra bit if ( ( ( zip_header.bit_flag & 4 ) >> 2 ) == 1 ) { is_descr = true; } else { is_descr = false; f_entr.pack_size[ f_entr.file_count - 1] = zip_header.pack_size; f_entr.unpack_size[ f_entr.file_count - 1] = zip_header.unpack_size; } //---Get current file name ArrayResize( f_name, zip_header.fn_len ); f_entr.zip_fnames[f_entr.file_count - 1] = ""; uint fn_res = FileReadArray( handle, f_name, 0, int( zip_header.fn_len ) ); for ( int i = 0; i < int( fn_res ); i++ ) f_entr.zip_fnames[f_entr.file_count - 1] += CharToString( f_name[i] ); //---Store offset f_entr.zip_offsets[ f_entr.file_count - 1] = FileTell( handle ); //---Get descriptor, if present if ( is_descr ) { uint sign; while ( FileTell( handle ) < ( file_size - uint_size ) ) { sign = FileReadInteger( handle, uint_size ); if ( ( sign == 0x04034b50 ) || ( sign == 0x02014b50 ) ) { //---Seek back for read descriptor FileSeek( handle, -( uint_size + descr_size ), SEEK_CUR ); FileReadStruct( handle, data_descriptor ); f_entr.pack_size[ f_entr.file_count - 1] = data_descriptor.pack_size; f_entr.unpack_size[ f_entr.file_count - 1] = data_descriptor.unpack_size; //---Align file positon to new file FileSeek( handle, f_entr.zip_offsets[f_entr.file_count - 1] + f_entr.pack_size[ f_entr.file_count - 1], SEEK_SET ); break; } } } else //---Seek to new file FileSeek( handle, f_entr.pack_size[ f_entr.file_count - 1 ], SEEK_CUR ); } } } else { Print( "Invalid zip file size!" ); } } else { Print( "Invalid zip file handle!" ); } return( f_entr.file_count ); } //+------------------------------------------------------------------+ //| Get file from ZIP file | //+------------------------------------------------------------------+ bool GetFile( const int handle, const ulong offset, const uint pack_size, const uint unp_size, uchar &unp_data[] ) { if ( ( handle != INVALID_HANDLE ) && ( pack_size != 0 ) && ( unp_size != 0 ) ) { uchar pack_data[]; ArrayResize( pack_data, pack_size + hdr_size ); FileSeek( handle, offset, SEEK_SET ); uint ar_res = FileReadArray( handle, pack_data, hdr_size, pack_size ); if ( ar_res == pack_size ) { pack_data[0] = 0x78; pack_data[1] = 0x5E; ResetLastError(); uint dec_res = CryptDecode( CRYPT_ARCH_ZIP, pack_data, key, unp_data ); if ( dec_res == unp_size ) { return( true ); } else { Print( "Last error = ", GetLastError() ); } } } return( false ); }Callouts:
Vasiliy!
Didn't wait for a reply from you. (don't know if MQ had time to implement changes in build 1100)
Decoder:
Callouts:
Mikalas, is it possible to make the decoding and encoding code as a library with a check for unpacking completion?
Sure. That's what I'll be doing in the near future. There will be packing/unpacking. Adding and removing files from archive. The main thing is that CryptDecode doesn't fail.
Vasiliy!
Is ZIP packaging necessary?
I don't think it's worth it.
1. If you want to build a database based on ZIP, it won't be relevant.
Soon MQ will add standard functions to work with the database (Renat did a survey).
2. You can't solve the problem with file sizes larger than uint.
There should be ZIP64 compression for ulong sizes.
3. You don't know exactly how MT5 data is compressed
It is possible that large files ( even uint size ) will
large files ( even uint size ) will be compressed for HOURS!
4. To "cram" files into an archive, you will have to keep quite a lot of information in
a lot of information in memory - there will be HARDWARE!
I strongly advise against it....
I am probably thinking the old-fashioned way, but for me archiving is interesting to speed up data transfer on the Internet. Locally with large hard drives the file size loses its relevance, and the database will be for a limited range of people, and it is costly to implement, as it will require additional knowledge from the programmer and the user.
It is true that zip is considerably inferior to rar in the compression, especially of text - which is a little sad.
Vasiliy!
Is ZIP packaging necessary?
I don't think it's worth it.
I think otherwise.
1. If you want to build a database based on ZIP, it won't be relevant.
Soon MQ will add standard functions to work with database (Renat did a survey)
No, zip is not interesting as an alternative to DB. That's not why it's worth bothering with zip.
2. You cannot solve the problem with file sizes larger than uint.
There must be ZIP64 compression for ulong size.
I have no goal to create an analogue of WinZip or WinRar. Only the most basic archives. No one will even think of compressing files larger than 4 GB.
3. You don't know how exactly MT5 data is compressed.
It's possible that large files ( even uint size) will be
even uint size) will be compressed for HOUR!
4. To "cram" files into an archive, you will have to keep quite a lot of information in
a lot of information in memory - there will be HARDWARE!
I strongly advise against....
1. MQ doesn't do slow functions, and if it does, it optimizes them in a timely manner, based on requests from servicedesk. I'm somehow confident that CryptEncode will fly.
2. There won't be any slowdowns. You do not know what a modern MQL5 is capable of in terms of performance. Today's computers have too much memory. To download and pack a file of a couple of hundreds of megabytes is a piece of cake, and you don't need more. For these are other tasks.
Vasiliy!
Is ZIP packaging necessary?
I don't think it's worth it.
I think otherwise.
No, zip is not interesting as an alternative to DB. That's not why it's worth bothering with zip.
I have no goal to create an analogue of WinZip or WinRar. Only the most basic archives. No one would even think of compressing files larger than 4 GB.
1. MQ doesn't do slow features, and if it does, it will optimize their work in a timely manner, based on requests from the service desk. I'm somehow confident that CryptEncode will fly.
2. There won't be any slowdowns. You do not know what a modern MQL5 is capable of in terms of performance. Today's computers have too much memory. To download and pack a file of a couple of hundreds of megabytes is a piece of cake, and you don't need more. For these are other tasks.
I am probably thinking the old-fashioned way, but for me archiving is interesting to speed up data transfer on the Internet. Locally with large hard drives the file size loses its relevance, and the database will be for a limited range of people, and it is costly to implement, as it will require additional knowledge from the programmer and the user.
True zip is significantly inferior in compression, especially text, rar - which is a little sad.
Quite true. Communicating with a third-party server via WebRequest will be much faster using the packaging of sent information. This is another idea why reloading the packaging is a good solution.
However, zip is significantly inferior to rar in compression, especially of text - which is a bit unfortunate.
Appreciate what we have been given. Believe me, the ability to work with the most popular compression format covers 90% of all tasks. 80% of redundancy is successfully eliminated with zip. What comes next is chasing parrots which nobody needs.
Good riddance!