Base64 file encoding

[zabolyx]

I'm working on the getting an image file to open in OOoBASIC and then read and encode the file as Base64 for use in another file.

Here's the concept... which I'm implementing... still in development


Open file

conversionstring = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"

For bytes = 1 to len(file) Step 3

Read byte1 from file ' I'm thinking that this can be done with GET but not sure how to limit it to 1 byte at a time
Read byte2 from file
Read byte3 from file

data = CByte(byte1) & CByte(byte2) & CByte(byte3)

For 6bit = 1 to 24 step 6

calculate decimal number from MID(data, 6bit, 6) ' i'm currently using a few if statements to add this value up - saved as 6bitvalue

sWriteString = sWriteString & MID(conversionstring, 6bitvalue, 1)

If len(sWriteString) = 64 Then

write sWriteString to email file at iEmailRow

reset sWriteString = ""

increment the row to write the data to
iEmailRow = iEmailRow + 1

End If

Next 6bit

Next bytes

[zabolyx]

Here is what I've got... I think I'm making this too complicated...

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integert, sFilePath as String, sFileName as String)
	
	sFilePath = sFilePath & sFileName
	
	'store the file number that is being used
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'store the bytes as a string to process
	Dim sDataBits as String
	
	'store the created string from the conversion for writing
	Dim sWriteString as String
	sWriteString = ""
	
	'store the value from the bit conversion
	Dim iBitNumber as Integer
	
	'open the file to encode as binanry data
	Open sFilePath For Binary as Read as iFileNumber
	
	'store the constant for conversion to base64
	Dim sConversion as String
	
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	'count through the file 3 bytes at a time
	For iChunk = 1 to FileLen(sFilePath) Step 3
		
		'get tehe 3 bytes from the file
		Get iFileNumber, iChunk, sByte1
		Get iFileNumber, iChunk + 1, sByte2
		Get iFileNumber, iChunk + 2, sByte3
		
		'convert the bytes to a string
		sDataBits = CByte(sByte1) & CByte(sByte2) & CByte(sByte3)
			
		'count through the data bits to convert to base64	
		For i6Bit = 1 to 24 Step 6
		
			iBitNumber = 0

			If MID(MID(sDataBits, i6Bit,1),1,1) Then iBitNumber = iBitNumber + 32
			If MID(MID(sDataBits, i6Bit,1),2,1) Then iBitNumber = iBitNumber + 16
			If MID(MID(sDataBits, i6Bit,1),3,1) Then iBitNumber = iBitNumber + 8
			If MID(MID(sDataBits, i6Bit,1),4,1) Then iBitNumber = iBitNumber + 4
			If MID(MID(sDataBits, i6Bit,1),5,1) Then iBitNumber = iBitNumber + 2
			If MID(MID(sDataBits, i6Bit,1),6,1) Then iBitNumber = iBitNumber + 1
			
			sWriteString = sWriteString & MID(sConversion, iBitNumber, 1)
			
			If Len(sWriteString) = 64 Then 
			
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
			
				iEmailRow = iEmailRow + 1
			
				sWtiteString = ""
				
			End If					
			
		'next segment to convert
		Next i6Bit		
		
	'on to getting the next 3 bytes
	Next iChunk
		
End Sub

[Charlie Young]

There is a Byte data type, though it doesn't seem to be very well documented. Use of that should simplify your code somewhat, as you wouldn't need the CByte stuff. The giveaway there is that there is a CByte function, which wouldn't make much sense if one couldn't declare a Byte variable!

[zabolyx]

OK... but how exactly would that help me. I need to combine the 3 bytes so that I can grab the data in 6bit segments to process.

Or would it be a matter of getting the bytes then shoving that into a string... so more or less just skipping the need for the CByte... OK that could work out

Also do you recommend setting up the opened file for random and set the record size to three bytes (I'm thinking that might work) but at the end of the file if I have fewer bytes would that cause me issues). I haven't messed with random access reading in forever. I usually use Calc to create a text file from a macro and save myself some hassle.

I'm supprised I've not seen any code that is from the ground up on Base64 conversions. I found several examples using pre-built API's or libraries for VB. And there is nothing that I've found in the OOo forums.. this one or the other on this subject.Once this code is done I'm throwing this routine into the snippets.I'm this close || to having my project done to a 1.0 version.

[zabolyx]

Started testing my code to see what I have going on (because at the moment... I'm not sure what I'm doing and sometimes this helps).

Getting an error on the first GET statement

Get iFileNumnber, iChunk, sByte1

DEVICE I/O ERROR

Now from what I've read in the help about Get is that I tell it which file (iFileNumber = 1 which I opened the image with), tell it which byte to get the data from (iChunk = 1 which is the start of the file) and save that data into a string (sByte1).

I used teh FreeFile statement to assigne iFileNumber so that I can't have a conflict with that (I should have had an issue with the OPEN).

[Charlie Young]

Started testing my code to see what I have going on (because at the moment... I'm not sure what I'm doing and sometimes this helps).

Getting an error on the first GET statement

Get iFileNumnber, iChunk, sByte1

DEVICE I/O ERROR

Now from what I've read in the help about Get is that I tell it which file (iFileNumber = 1 which I opened the image with), tell it which byte to get the data from (iChunk = 1 which is the start of the file) and save that data into a string (sByte1).

I used teh FreeFile statement to assigne iFileNumber so that I can't have a conflict with that (I should have had an issue with the OPEN).

Is sByte1 a String? It might be trying to read the whole file in one gulp. The notion of a random file with a 3 byte record length is interesting, but you should be able to read a byte at a time if sByte1 is declared as a Byte, though I have not gotten that far with my experiments yet.

I have written a function which seems to work. It takes a 3 byte string and returns a 4 byte string. It works by putting the three bytes into the rightmost 3 bytes of a Long, then masking off the 6 bit sections of the 3 bytes.

Code: Select all

Function Base64Convert(StringIn As String) As String
	Dim sConversion as String
   	Dim Masks(3) As Long
   	Dim Bytes3 As Long
   	   	
   	Masks(0) = &H00FC0000  	'Binary 00000000111111000000000000000000
   	Masks(1) = &H0003F000	'Binary 00000000000000111111000000000000
   	Masks(2) = &H00000FC0	'Binary 00000000000000000000111111000000
   	Masks(3) = &H0000003F	'Binary 00000000000000000000000000111111
   	
   	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	Bytes3 = 256*256*Asc(Mid(StringIn,1,1)) + 256*Asc(Mid(StringIn,2,1)) + Asc(Mid(StringIn,3,1))
	
	Base64Convert = Mid(sConversion,(Bytes3 AND Masks(0))/(4*256*256)+1, 1) &_
					Mid(sConversion,(Bytes3 AND Masks(1))/(4*4*256)+1, 1) &_
					Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4)+1, 1) &_
					Mid(sConversion,(Bytes3 AND Masks(3))+1, 1) 
End Function

I'm going to do some more experiments. I've just worked with literal strings so far. I'll try it on a file next.

[Charlie Young]

My previous version of the Base64Convert function ignored the issue of padding if the last string sent to it only had 1 or 2 bytes. This version, using "=" as the pad character, handles that situation.

Code: Select all

Function Base64Convert(StringIn As String) As String
	Dim sConversion as String
	Dim Masks(3) As Long
	Dim Bytes3 As Long
	Dim l As Integer
   	
	Masks(0) = &H00FC0000	'Binary 00000000111111000000000000000000
	Masks(1) = &H0003F000	'Binary 00000000000000111111000000000000
	Masks(2) = &H00000FC0	'Binary 00000000000000000000111111000000
	Masks(3) = &H0000003F	'Binary 00000000000000000000000000111111
   	
	l = Len(StringIn)
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	if l = 3 then
		Bytes3 = 256*256*Asc(Mid(StringIn,1,1)) + 256*Asc(Mid(StringIn,2,1)) + Asc(Mid(StringIn,3,1))
		
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(0))/(4*256*256) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(1))/(4*4*256) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1) 
	elseif l = 2 then
		Bytes3 = 256*Asc(Mid(StringIn,1,1)) + Asc(Mid(StringIn,2,1))
		Bytes3 = 4*Bytes3 ' Shift bits left 2 to give 18 bits to mask
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(1))/(4*4*256) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1) &_
						"=" 
	else
		Bytes3 = Asc(Mid(StringIn,1,1))
		Bytes3 = 16*Bytes3 ' Shift bits left 4 to give 12 bits to mask
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) &_
						Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1) &_
						"=="
	endif
	
End Function

I'm also finding that opening the file as Binary and reading it one byte at a time with Get does work.

[zabolyx]

Here is what I got... I'm getting ready to test... your code is beautiful like always....

I'll report on what happens... and then post the entire working code for generating the eml files. Just a shame that I can't get OOo to directly email these but this will give me an excuse to instal Thunderbird on everyone's computers

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integer, sFilePath as String, sFileName as String, oEmailSheet as Object)
	
	'combine the file path and file name
	sFilePath = ConvertToURL(sFilePath & "/" & sFileName)
	
	'store the byte from the opened file
	Dim sGotByte as Byte
	
	'store the number of the file being opened
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'open the file to be read as binary access
	Open sFilePath For Binary as iFileNumber
	
	'store the location of current byte being pulled
	Dim iByte as Long
	
	'store the combined bytes for conversion
	Dim sByteString as String
	
	'store the Base64 converted string 
	Dim sWriteString as String
	
	'store the lenght of the file being processed
	Dim iFileLength as Long
	iFileLength = FileLen(sFilePath)
	
	'count through the bytes in the file to process
	For iByte = 1 to iFileLength
		
		'retreive the current byte
		Get iFileNumber, iByte, sGotByte
		
		'combine the bytes into a string for processing
		sByteString = sByteString + sGotByte
		
		'increment the loop counter
		iRetreivedBytes = iRetreivedBytes + 1
		
		'check if three bytes have been combined
		If iRetreivedBytes = 3 OR iByte = iFileLength Then
			
			'combine the converted string for writing
			sWriteString = sWriteString & Base64Convert(sByteString)
			
			'increment the loop counter
			iRetreivedBytes = iRetreivedBytes + 1
			
			'check if the lenght of the write string is ready for writing to the file
			If Len(sWriteString) = 64 Then
				
				'write the string into the EML file
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
				
				'set the new iEmailRow counter
				iEmailRow = iEmailRow +  1
											
			'end the current if statement
			End If
			
		'end the current if statement
		End If 
				
	'next byte to read
	Next iByte
		
End Sub





Function Base64Convert(StringIn As String) As String

	Dim sConversion as String
	Dim Masks(3) As Long
	Dim Bytes3 As Long
	Dim l As Integer
	      
	Masks(0) = &H00FC0000   'Binary 00000000111111000000000000000000
	Masks(1) = &H0003F000   'Binary 00000000000000111111000000000000
	Masks(2) = &H00000FC0   'Binary 00000000000000000000111111000000
	Masks(3) = &H0000003F   'Binary 00000000000000000000000000111111
	      
	l = Len(StringIn)
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	If l = 3 Then
	
		Bytes3 = 256*256*Asc(Mid(StringIn,1,1)) + 256*Asc(Mid(StringIn,2,1)) + Asc(Mid(StringIn,3,1))
		      
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(0))/(4*256*256) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(1))/(4*4*256) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1)
	
	ElseIf l = 2 Then
	   
		Bytes3 = 256 * Asc(Mid(StringIn,1,1)) + Asc(Mid(StringIn,2,1))
		      
		Bytes3 = 4 * Bytes3 ' Shift bits left 2 to give 18 bits to mask
		      
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(1))/(4*4*256) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1) & "="
	
	Else
	
		Bytes3 = Asc(Mid(StringIn,1,1))
		      
		Bytes3 = 16 * Bytes3 ' Shift bits left 4 to give 12 bits to mask
		      
		Base64Convert = Mid(sConversion,(Bytes3 AND Masks(2))/(4*4*4) + 1, 1) & Mid(sConversion,(Bytes3 AND Masks(3)) + 1, 1) & "=="
	      
	EndIf
   
End Function

[zabolyx]

OK... I'm testing and I'm finding that it seems to skipp the first 2 bytes of the file... not sure why but I did find a thread in the forum concerning this... the user ended up getting around this issue using SimpleFileAccess and pulling the data as a stream... Not sure how that eill work for me.

Also I'm not sure that I'm passing the data correctly to the Function you wrote... I'm getting the data as bytes and then combining those as I would a string. Then passing that on through... so my first string I had being snet was 255156255 as the string. which did not work. Do I need to convert that to a binary number to have the function process it?

[Charlie Young]

I'm using a loop like this to test, where file #1 (Urlin) is the input file, and #2 is the encoded output. Now, I've been testing this on both plain text and graphic file input, and the plain text works fine, but I'm having problems with the graphic file. I took the next logical step and wrote a decode function, which I'll post below even though I don't completely trust it yet, though it does seem to work on text. Note that FileLen(UrlIn) - 2 is just there to skip a CRLF at the end of the file.

Base64Convert just expects to receive a string of 1-3 bytes, and note that I do use Chr() to put the input byte to the string. All but the final string should be 3 bytes, and the last one can be 1, 2, or 3.

Code: Select all

For i = 1 to FileLen(UrlIn) - 2
		Get #1, i, InByte
		InString = InString & Chr(InByte)
		if Len(InString) = 3 then
			Print #2,Base64Convert(InString);
			InString = ""
		endif
	next i
	if Len(Instring) > 0 then
		Print #2,Base64Convert(InString)
	endif

The Decode function. it expects a string of length 4, which may or may not terminate in "=" or "==" which should only be present on the final call.

Code: Select all

Function Base64Decode(StringIn As String) As String
	Dim sConversion as String
	Dim Masks(2) As Long
	Dim Bits24 As Long
	Dim l As Integer
   	
	Masks(0) = &H00FF0000	
	Masks(1) = &H0000FF00	
	Masks(2) = &H000000FF	
	   	
	l = Len(StringIn)
	
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	if (Right(StringIn,2) <> "==") and (Right(StringIn,1) <> "=") then
		Bits24 = 64*64*64*(InStr(1,sConversion,Mid(StringIn,1,1),0) - 1) +_
				64*64*(InStr(1,sConversion,Mid(StringIn,2,1),0) - 1) + _
				64*(InStr(1,sConversion,Mid(StringIn,3,1),0) - 1) + _
				InStr(1,sConversion,Mid(StringIn,4,1),0) - 1
		Base64Decode = Chr((Bits24 AND Masks(0))/(256 * 256)) & Chr((Bits24 AND Masks(1))/256) & Chr(Bits24 AND Masks(2))
						 
	elseif (Right(StringIn,1) = "=") and (Right(StringIn,2) <> "==") then 
		Bits24 = 64*64*64*(InStr(1,sConversion,Mid(StringIn,1,1),0) - 1) +_
				64*64* (InStr(1,sConversion,Mid(StringIn,2,1),0) - 1) + _
				64 * InStr(1,sConversion,Mid(StringIn,3,1),0) - 1
				
		Base64Decode = Chr((Bits24 AND Masks(0))/(256*256)) & Chr((Bits24 AND Masks(1))/256) 
	else
		Bits24 = 64*64*64*(InStr(1,sConversion,Mid(StringIn,1,1),0) - 1) +_
				64*64*(InStr(1,sConversion,Mid(StringIn,2,1),0) - 1) 
								
		Base64Decode = Chr((Bits24 AND Masks(0))/(256 * 256))
	endif
	
End Function

[zabolyx]

I think the reason the conversion is not working on the image file is that CHR is only capable of displaying the characters that are available to display. ASCII 7 is a bell sound (for making a beep on the PC speaker) and I'm sure there are others that do not convert to strings when processed... this is causing me all sorts of issues.... The blank characters seem to be recognized as just missing so the file gets filled with lots of ='s everywhere.

If we convert the string into 1's and 0's before processing we could avoid this... as we are starting out with the byte value of 0 - 255. Converting that into a binary string would allow us to process as well... just a rewriting of the converter would be needed.

I'll see what I can come up with over the weekend. I also noted that I did not have the first 2 bytes skipped on the latest runs of the routine when converting to a CHR... Not sure what is different to do that. Whatever... I'll plink away at it this weekend as well... I've got to get this working before the new year... as I'd like to have my team using it to track their progress this year.

You have been amazing Sir Charles.

[zabolyx]

Her is what I have.. but it won't find the first character in the 3 byte segment.. it gets the value but not the character from the sConversion string...

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integer, sFilePath as String, sFileName as String, oEmailSheet as Object)
	
	'combine the file path and file name
	sFilePath = ConvertToURL(sFilePath & "/" & sFileName)
	
	'store the byte from the opened file
	Dim sGotByte as Byte
	
	'store the number of the file being opened
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'open the file to be read as binary access
	Open sFilePath For Binary as iFileNumber
	
	'store the location of current byte being pulled
	Dim iByte as Long
	
	'store the combined bytes for conversion
	Dim sByteString as String
	
	'store the Base64 converted string 
	Dim sWriteString as String
	
	'store the lenght of the file being processed
	Dim iFileLength as Long
	iFileLength = FileLen(sFilePath)
	
	'store the individual bits from the Byte for conversion
	Dim sBit(8) as String
	
	'clear the bit array
	For a = 1 to 8
		sBit(a) = 0
	Next a
	
	'count through the bytes in the file to process
	For iByte = 1 to iFileLength
		
		'retreive the current byte
		Get iFileNumber, iByte, sGotByte
		
		'parse out the byte into bits
		If sGotByte > 128 Then
			sBit(1) = 1
			sGotByte = sGotByte - 128
		End If
		If sGotByte > 64 Then
			sBit(2) = 1
			sGotByte = sGotByte - 64
		End If
		If sGotByte > 32 Then
			sBit(3) = 1
			sGotByte = sGotByte - 32
		End If
		If sGotByte > 16 Then
			sBit(4) = 1
			sGotByte = sGotByte - 16
		End If
		If sGotByte > 8 Then
			sBit(5) = 1
			sGotByte = sGotByte - 8
		End If
		If sGotByte > 4 Then
			sBit(6) = 1
			sGotByte = sGotByte - 4
		End If
		If sGotByte > 2 Then
			sBit(7) = 1
			sGotByte = sGotByte - 2
		End If
		If sGotByte > 1 Then
			sBit(8) = 1
			sGotByte = sGotByte - 1
		End If
				
		'combine the bytes into a string for processing
		For a = 1 to 8
			sByteString = sByteString & sBit(a)
		Next a

'		'combine the bytes into a string for processing
'		sByteString = sByteString & sGotByte
		
		'increment the loop counter
		iRetreivedBytes = iRetreivedBytes + 1
		
		'check if three bytes have been combined
		If iRetreivedBytes = 3 OR iByte = iFileLength Then
			
			'combine the converted string for writing
			sWriteString = sWriteString & Base64Convert(sByteString)
			
			'reset the loop counter
			iRetreivedBytes = 0
			
			'check if the lenght of the write string is ready for writing to the file
			If Len(sWriteString) = 64 Then
				
				'write the string into the EML file
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
				
				'set the new iEmailRow counter
				iEmailRow = iEmailRow +  1
				
				'clear the converted string for writing
				sWriteString = ""
				
			'end the current if statement
			End If
			
		'end the current if statement
		End If 
				
	'next byte to read
	Next iByte
		
End Sub





Function Base64Convert(sIncoming As String) As String
	
	'store the length of the incoming string
	Dim iLength as Integer
	iLength = Len(sIncoming)
	
	'store the 6 bit segment to work with
	Dim sSegment as String
	
	'store the converted character
	Dim sBase64 as String
	
	'store the 6 bit value
	Dim i6Bit as Integer
	i6Bit = 0
	
	'store the conversion string
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	'count throught the string 6 bits at a time for processing
	For iChunk = 1 to iLength - 1 Step 6
		
		'get the 6 bit segment to process
		sSegment = MID(sIncoming, iChunk, 6)
		
		'convert to a number to be used
		If MID(sSegment, 1, 1) = "1" Then i6Bit = i6Bit + 32
		If MID(sSegment, 2, 1) = "1" Then i6Bit = i6Bit + 16
		If MID(sSegment, 3, 1) = "1" Then i6Bit = i6Bit + 8
		If MID(sSegment, 4, 1) = "1" Then i6Bit = i6Bit + 4
		If MID(sSegment, 5, 1) = "1" Then i6Bit = i6Bit + 2
		If MID(sSegment, 6, 1) = "1" Then i6Bit = i6Bit + 1
		i6Bit = i6Bit + 1
		
		'get the item from the conversion string
		 sBase64 = sBase64 & MID(sConversion, i6Bit, 1)
		 
		 i6Bit = 0
				
	'next 6 bit chuck to process
	Next iChunk

	If iLength = 16 Then 
		sBase64 = sBase64 & "="
	EndIf
		
	If iLength = 8 Then 
		sBase64 = sBase64 & "=="
	EndIf
	
	'return the data
	Base64Convert = sBase64
	
End Function

I'm sure it isn't the prettiest code but it seems to me that this should get around the non-printable characters that the other method refuses to deal with. I think I did that padding correctly but not sure. It doesn't need to be on the 6bit... 8 is correct right?

I convert the 3 byte values that are read from the file into binary string equivalents and then send that to the converter. There it grabs the 4 6bit chunks and calculates a value and uses that to search the conversion string. As far as I can tell this should work.... Just need to get teh first character to convert.

[zabolyx]

Bug checking and found a few... fist of left a few characters out of the conversion string... that is why the first character wasn't coming up...

Forgot to clear the sByteString when it needed to so it just kept growing.. made a mess.. those corrections are below.

Now I'm getting the first 4 characters back correctly... and then every character after those are /'s... (63). And I do mean every.

She's getting there... just not quite as far as I'd like to have been... so she's going home with me to get a working over.

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integer, sFilePath as String, sFileName as String, oEmailSheet as Object)
	
	'combine the file path and file name
	sFilePath = ConvertToURL(sFilePath & "/" & sFileName)
	
	'store the byte from the opened file
	Dim sGotByte as Byte
	
	'store the number of the file being opened
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'open the file to be read as binary access
	Open sFilePath For Binary as iFileNumber
	
	'store the location of current byte being pulled
	Dim iByte as Long
	
	'store the combined bytes for conversion
	Dim sByteString as String
	
	'store the Base64 converted string 
	Dim sWriteString as String
	
	'store the lenght of the file being processed
	Dim iFileLength as Long
	iFileLength = FileLen(sFilePath)
	
	'store the individual bits from the Byte for conversion
	Dim sBit(8) as String
	
	'clear the bit array
	For a = 1 to 8
		sBit(a) = 0
	Next a
	
	'count through the bytes in the file to process
	For iByte = 1 to iFileLength
		
		'retreive the current byte
		Get iFileNumber, iByte, sGotByte
				
		'parse out the byte into bits
		If sGotByte > 128 Then
			sBit(1) = 1
			sGotByte = sGotByte - 128
		End If
		If sGotByte > 64 Then
			sBit(2) = 1
			sGotByte = sGotByte - 64
		End If
		If sGotByte > 32 Then
			sBit(3) = 1
			sGotByte = sGotByte - 32
		End If
		If sGotByte > 16 Then
			sBit(4) = 1
			sGotByte = sGotByte - 16
		End If
		If sGotByte > 8 Then
			sBit(5) = 1
			sGotByte = sGotByte - 8
		End If
		If sGotByte > 4 Then
			sBit(6) = 1
			sGotByte = sGotByte - 4
		End If
		If sGotByte > 2 Then
			sBit(7) = 1
			sGotByte = sGotByte - 2
		End If
		If sGotByte > 1 Then
			sBit(8) = 1
			sGotByte = sGotByte - 1
		End If
				
		'combine the bytes into a string for processing
		For a = 1 to 8
			sByteString = sByteString & sBit(a)
		Next a

'		'combine the bytes into a string for processing
'		sByteString = sByteString & sGotByte
		
		'increment the loop counter
		iRetreivedBytes = iRetreivedBytes + 1
		
		'check if three bytes have been combined
		If iRetreivedBytes = 3 OR iByte = iFileLength Then
			
			'combine the converted string for writing
			sWriteString = sWriteString & Base64Convert(sByteString)
			
			'reset the loop counter
			iRetreivedBytes = 0
			
			'reset the byte string
			sByteString = ""
			
			'check if the lenght of the write string is ready for writing to the file
			If Len(sWriteString) = 64 Then
				
				'write the string into the EML file
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
				
				'set the new iEmailRow counter
				iEmailRow = iEmailRow +  1
				
				'clear the converted string for writing
				sWriteString = ""
				
			'end the current if statement
			End If
			
		'end the current if statement
		End If 
				
	'next byte to read
	Next iByte
		
End Sub





Function Base64Convert(sIncoming As String) As String
	
	'store the length of the incoming string
	Dim iLength as Integer
	iLength = Len(sIncoming)
	
	'store the 6 bit segment to work with
	Dim sSegment as String
	sSegment = ""
	
	'store the converted character
	Dim sBase64 as String
	sBase64 = ""
	
	'store the 6 bit value
	Dim i6Bit as Integer
	i6Bit = 0
	
	'store the conversion string
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	'count throught the string 6 bits at a time for processing
	For iChunk = 1 to iLength - 1 Step 6
		
		'get the 6 bit segment to process
		sSegment = MID(sIncoming, iChunk, 6)
		
		'convert to a number to be used
		If MID(sSegment, 1, 1) = "1" Then i6Bit = i6Bit + 32
		If MID(sSegment, 2, 1) = "1" Then i6Bit = i6Bit + 16
		If MID(sSegment, 3, 1) = "1" Then i6Bit = i6Bit + 8
		If MID(sSegment, 4, 1) = "1" Then i6Bit = i6Bit + 4
		If MID(sSegment, 5, 1) = "1" Then i6Bit = i6Bit + 2
		If MID(sSegment, 6, 1) = "1" Then i6Bit = i6Bit + 1
		i6Bit = i6Bit + 1
		
		'get the item from the conversion string
		 sBase64 = sBase64 & MID(sConversion, i6Bit, 1)
		 
		 i6Bit = 0
				
	'next 6 bit chuck to process
	Next iChunk

	If iLength = 16 Then 
		sBase64 = sBase64 & "="
	EndIf
		
	If iLength = 8 Then 
		sBase64 = sBase64 & "=="
	EndIf
	
	'return the data
	Base64Convert = sBase64
	
	sBase64 = ""
	
End Function

[zabolyx]

OK.. wasn't clearing out the sBit array and there for eventually all of the bits would become 1's... Ya I know a basic oversite

A few other changes made and it seems to be working... But I'm not getting the image to show up in the EML file when pulled into Thunderbird... I think this might be an issue with the EML file formatting... but the EML file doesn't completely break (so I'm no worse off).

Here's what I have at the moment.

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integer, sFilePath as String, sFileName as String, oEmailSheet as Object)
	
	'combine the file path and file name
	sFilePath = ConvertToURL(sFilePath & "/" & sFileName)
	
	'store the byte from the opened file
	Dim sGotByte as Byte
	
	'store the number of the file being opened
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'open the file to be read as binary access
	Open sFilePath For Binary as iFileNumber
	
	'store the location of current byte being pulled
	Dim iByte as Long
	
	'store the combined bytes for conversion
	Dim sByteString as String
	
	'store the Base64 converted string 
	Dim sWriteString as String
	
	'store the lenght of the file being processed
	Dim iFileLength as Long
	iFileLength = FileLen(sFilePath)
	
	'store the individual bits from the Byte for conversion
	Dim sBit(8) as String
	
	'clear the bit array
	For a = 1 to 8
		sBit(a) = 0
	Next a
	
	'count through the bytes in the file to process
	For iByte = 1 to iFileLength
		
		'retreive the current byte
		Get iFileNumber, iByte, sGotByte
				
		'parse out the byte into bits
		If sGotByte > 128 Then
			sBit(1) = 1
			sGotByte = sGotByte - 128
		End If
		If sGotByte > 64 Then
			sBit(2) = 1
			sGotByte = sGotByte - 64
		End If
		If sGotByte > 32 Then
			sBit(3) = 1
			sGotByte = sGotByte - 32
		End If
		If sGotByte > 16 Then
			sBit(4) = 1
			sGotByte = sGotByte - 16
		End If
		If sGotByte > 8 Then
			sBit(5) = 1
			sGotByte = sGotByte - 8
		End If
		If sGotByte > 4 Then
			sBit(6) = 1
			sGotByte = sGotByte - 4
		End If
		If sGotByte > 2 Then
			sBit(7) = 1
			sGotByte = sGotByte - 2
		End If
		If sGotByte > 1 Then
			sBit(8) = 1
			sGotByte = sGotByte - 1
		End If
				
		'combine the bytes into a string for processing
		For a = 1 to 8
			sByteString = sByteString & sBit(a)
			sBit(a) = 0
		Next a

		'clear the bits array
		For a = 1 to 8
			sBit(a) = 0
		Next a

'		'combine the bytes into a string for processing
'		sByteString = sByteString & sGotByte
		
		'increment the loop counter
		iRetreivedBytes = iRetreivedBytes + 1
		
		'check if three bytes have been combined
		If iRetreivedBytes = 3 OR iByte = iFileLength Then
			
			'combine the converted string for writing
			sWriteString = sWriteString & fBase64Convert(sByteString)
			
			'reset the loop counter
			iRetreivedBytes = 0
			
			'reset the byte string
			sByteString = ""
			
			'check if the lenght of the write string is ready for writing to the file
			If Len(sWriteString) = 64 Then
				
				'write the string into the EML file
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
				
				'set the new iEmailRow counter
				iEmailRow = iEmailRow +  1
				
				'clear the converted string for writing
				sWriteString = ""
				
			'end the current if statement
			End If
			
		'end the current if statement
		End If 
				
	'next byte to read
	Next iByte
		
End Sub





Function fBase64Convert(sIncoming As String) As String
	
	'store the length of the incoming string
	Dim iLength as Integer
	iLength = Len(sIncoming)
	
	'store the 6 bit segment to work with
	Dim sSegment as String
	sSegment = ""
	
	'store the converted character
	Dim sBase64 as String
	sBase64 = ""
	
	'store the 6 bit value
	Dim i6Bit as Integer
	i6Bit = 0
	
	'store the conversion string
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	'count throught the string 6 bits at a time for processing
	For iChunk = 1 to iLength - 1 Step 6
		
		'get the 6 bit segment to process
		sSegment = MID(sIncoming, iChunk, 6)
		
		'convert to a number to be used
		If MID(sSegment, 1, 1) = "1" Then i6Bit = i6Bit + 32
		If MID(sSegment, 2, 1) = "1" Then i6Bit = i6Bit + 16
		If MID(sSegment, 3, 1) = "1" Then i6Bit = i6Bit + 8
		If MID(sSegment, 4, 1) = "1" Then i6Bit = i6Bit + 4
		If MID(sSegment, 5, 1) = "1" Then i6Bit = i6Bit + 2
		If MID(sSegment, 6, 1) = "1" Then i6Bit = i6Bit + 1
		i6Bit = i6Bit + 1
		
		'get the item from the conversion string
		 sBase64 = sBase64 & MID(sConversion, i6Bit, 1)
		 
		 i6Bit = 0
				
	'next 6 bit chuck to process
	Next iChunk

	If iLength = 16 Then 
		sBase64 = sBase64 & "="
	EndIf
		
	If iLength = 8 Then 
		sBase64 = sBase64 & "=="
	EndIf
	
	'return the data
	fBase64Convert = sBase64
	
	sBase64 = ""
	
End Function

[Charlie Young]

I think I have found the fundamental problem, and it's a nasty one: the Put statement does not write null characters Chr(0) to a binary file. Since graphics file tend to be full of these, that is very sticky. I'm trying to find a workaround using the API, but no luck so far.

I did write a Visual Basic 2010 program that does work. It uses the Base64 text file made with the Calc function, though I have modified it now to write each 4 byte string on a separate line, which makes the input easier. A small chunk looks like this:

Code: Select all

Qk3i
fgAA
AAAA
ADYA
AAAo
AAAA
oQAA
AEMA
AAAB
ABgA
AAAA
AKx+
AADf
DgAA
CA8A
AAAA
AAAA
AAAA
////
////
////
////

The VB program: the decode function is virtually identical to the OOo version, but the file manipulation is very different. The conversion sub is called from main passing the base64 text file name and the output file name (which may or may not be a graphics file). The full paths should be provided, but Url notation doesn't work in VB.

Code: Select all

Imports Microsoft.VisualBasic
Imports System.IO

Module Module1

    Sub Main()
        ConvertBase64File("Base64File.txt", "outputfile")
    End Sub

    Function Base64Decode(ByVal StringIn As String) As String
        Dim sConversion As String
        Dim Masks(2) As Long
        Dim Bits24 As Long
        
        Masks(0) = &HFF0000
        Masks(1) = &HFF00
        Masks(2) = &HFF

        sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
        If (Right(StringIn, 2) <> "==") And (Right(StringIn, 1) <> "=") Then
            Bits24 = 64 * 64 * 64 * (InStr(1, sConversion, Mid(StringIn, 1, 1), 0) - 1) + _
            64 * 64 * (InStr(1, sConversion, Mid(StringIn, 2, 1), 0) - 1) + _
            64 * (InStr(1, sConversion, Mid(StringIn, 3, 1), 0) - 1) + _
            InStr(1, sConversion, Mid(StringIn, 4, 1), 0) - 1
            Base64Decode = Chr((Bits24 And Masks(0)) / (256 * 256)) & Chr((Bits24 And Masks(1)) / 256) & Chr(Bits24 And Masks(2))
        ElseIf (Right(StringIn, 1) = "=") And (Right(StringIn, 2) <> "==") Then
            Bits24 = 64 * 64 * 64 * (InStr(1, sConversion, Mid(StringIn, 1, 1), 0) - 1) + _
            64 * 64 * (InStr(1, sConversion, Mid(StringIn, 2, 1), 0) - 1) + _
            64 * InStr(1, sConversion, Mid(StringIn, 3, 1), 0) - 1

            Base64Decode = Chr((Bits24 And Masks(0)) / (256 * 256)) & Chr((Bits24 And Masks(1)) / 256)
        Else
            Bits24 = 64 * 64 * 64 * (InStr(1, sConversion, Mid(StringIn, 1, 1), 0) - 1) + _
            64 * 64 * (InStr(1, sConversion, Mid(StringIn, 2, 1), 0) - 1)

            Base64Decode = Chr((Bits24 And Masks(0)) / (256 * 256))
        End If

    End Function

    Sub ConvertBase64File(ByVal Base64FileName As String, ByVal GraphicFileName As String)
        Dim aFileStream As FileStream
        Dim Base64FileReader As System.IO.StreamReader
        Dim sInputLine As String
        Dim DecodeStr As String
        Dim OutByte As Byte
        Dim i As Integer

        aFileStream = New FileStream(GraphicFileName, FileMode.OpenOrCreate, FileAccess.Write)
        Dim GraphicWriter As New BinaryWriter(aFileStream)
        Base64FileReader = System.IO.File.OpenText(Base64FileName)
        sInputLine = Base64FileReader.ReadLine()
        Do Until sInputLine Is Nothing
            DecodeStr = Base64Decode(sInputLine)
            For i = 1 To Len(DecodeStr)
                OutByte = Asc(Mid(DecodeStr, i, 1))
                GraphicWriter.BaseStream.WriteByte(OutByte)
            Next i
            sInputLine = Base64FileReader.ReadLine()

        Loop
        Base64FileReader.Close()
        GraphicWriter.Close()
    End Sub

End Module

[zabolyx]

OK... so I won't be able to write the decoded data to a file with the standard method. For this project not an issue but definitely a hindrance to the a Base64 encoder/decoder setup...

What about the data stream from the SimpleFileAccess? Would that prove useful?

As for my encoder for my email it seems to be encoding... but I still can't read it. I ran the image file through a base64 converter online and the resulting data is diffferent than what I'm getting... I'll have to dig into that a little more.

[zabolyx]

Here we go... I was dumb but then I saw the light... My decoder wasn't checking if a value was greater than AND equal to that value when decoding the byte value.

Here be the fixed code for the opening of the file and encoding it into base64... this does work with images and from what I've seen just about anything. I'm going to be furthering this process as part of my email engine to add the PDF attachment that I'm needing.

Code: Select all

Sub mMailMergeBase64 (iEmailRow as Integer, sFilePath as String, sFileName as String, oEmailSheet as Object)
	
	'combine the file path and file name
	sFilePath = ConvertToURL(sFilePath & "/" & sFileName)
	
	'store the byte from the opened file
	Dim sGotByte as Byte
	
	'store the number of the file being opened
	Dim iFileNumber as Integer
	iFileNumber = FreeFile
	
	'open the file to be read as binary access
	Open sFilePath For Binary as iFileNumber
	
	'store the location of current byte being pulled
	Dim iByte as Long
	
	'store the combined bytes for conversion
	Dim sByteString as String
	
	'store the Base64 converted string 
	Dim sWriteString as String
	
	'store the lenght of the file being processed
	Dim iFileLength as Long
	iFileLength = FileLen(sFilePath)
	
	'store the individual bits from the Byte for conversion
	Dim sBit(8) as String
	
	'clear the bit array
	For a = 1 to 8
		sBit(a) = 0
	Next a
	
	'count through the bytes in the file to process
	For iByte = 1 to iFileLength
		
		'retreive the current byte
		Get iFileNumber, iByte, sGotByte
		
		'parse out the byte into bits
		If sGotByte > 128 OR sGotByte = 128 Then
			sBit(1) = 1
			sGotByte = sGotByte - 128
		End If
		If sGotByte > 64 OR sGotByte = 64 Then
			sBit(2) = 1
			sGotByte = sGotByte - 64
		End If
		If sGotByte > 32 OR sGotByte = 32 Then
			sBit(3) = 1
			sGotByte = sGotByte - 32
		End If
		If sGotByte > 16 OR sGotByte = 16 Then
			sBit(4) = 1
			sGotByte = sGotByte - 16
		End If
		If sGotByte > 8 OR sGotByte = 8 Then
			sBit(5) = 1
			sGotByte = sGotByte - 8
		End If
		If sGotByte > 4 OR sGotByte = 4 Then
			sBit(6) = 1
			sGotByte = sGotByte - 4
		End If
		If sGotByte > 2 OR sGotByte = 2 Then
			sBit(7) = 1
			sGotByte = sGotByte - 2
		End If
		If sGotByte = 1 Then
			sBit(8) = 1
			sGotByte = sGotByte - 1
		End If
				
		'combine the bytes into a string for processing
		For a = 1 to 8
			sByteString = sByteString & sBit(a)
'			sBit(a) = 0
		Next a

		'clear the bits array
		For a = 1 to 8
			sBit(a) = 0
		Next a

		'increment the loop counter
		iRetreivedBytes = iRetreivedBytes + 1
		
		'check if three bytes have been combined
		If iRetreivedBytes = 3 OR iByte = iFileLength Then
			
			'combine the converted string for writing
			sWriteString = sWriteString & fBase64Convert(sByteString)
			
			'reset the loop counter
			iRetreivedBytes = 0
			
			'reset the byte string
			sByteString = ""
			
			'check if the lenght of the write string is ready for writing to the file
			If Len(sWriteString) = 76 OR iByte = iFileLength Then
				
				'write the string into the EML file
				oEmailSheet.getCellRangeByName("A" & iEmailRow).String = sWriteString
				
				'set the new iEmailRow counter
				iEmailRow = iEmailRow +  1
				
				'clear the converted string for writing
				sWriteString = ""
				
			'end the current if statement
			End If
			
		'end the current if statement
		End If 
				
	'next byte to read
	Next iByte
		
End Sub





Function fBase64Convert(sIncoming As String) As String
	
	'store the length of the incoming string
	Dim iLength as Integer
	iLength = Len(sIncoming)
	
	'store the 6 bit segment to work with
	Dim sSegment as String
	sSegment = ""
	
	'store the converted character
	Dim sBase64 as String
	sBase64 = ""
	
	'store the 6 bit value
	Dim i6Bit as Integer
	i6Bit = 0
	
	'store the conversion string
	sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
	
	'count throught the string 6 bits at a time for processing
	For iChunk = 1 to iLength - 1 Step 6
		
		'get the 6 bit segment to process
		sSegment = MID(sIncoming, iChunk, 6)
		
		'convert to a number to be used
		If MID(sSegment, 1, 1) = "1" Then i6Bit = i6Bit + 32
		If MID(sSegment, 2, 1) = "1" Then i6Bit = i6Bit + 16
		If MID(sSegment, 3, 1) = "1" Then i6Bit = i6Bit + 8
		If MID(sSegment, 4, 1) = "1" Then i6Bit = i6Bit + 4
		If MID(sSegment, 5, 1) = "1" Then i6Bit = i6Bit + 2
		If MID(sSegment, 6, 1) = "1" Then i6Bit = i6Bit + 1
		i6Bit = i6Bit + 1
		
		'get the item from the conversion string
		 sBase64 = sBase64 & MID(sConversion, i6Bit, 1)
		 
		 i6Bit = 0
				
	'next 6 bit chuck to process
	Next iChunk

	If iLength = 16 Then 
		sBase64 = sBase64 & "="
	EndIf
		
	If iLength = 8 Then 
		sBase64 = sBase64 & "=="
	EndIf
	
	'return the data
	fBase64Convert = sBase64
	
	sBase64 = ""
	
End Function

It is slower that I'd like it to be... so that is something I'd like to look into later. Any improvement in speed would be great. Of course a proper decoder would be great to go with this but that is out of the scope I've set for myself. Charlie feel free to continue on that process. As you stated the PUT statement doesn't like null characters as well CHR not liking the non-printable characters.

[RoryOF]

I haven't looked at your code, but I remind you that for optimisation the usual course is to take everything possible out of the processing loops.

[Charlie Young]

I've been trying the SimpleFileAccess approach, but all I've been able to do is write an output file with all null characters. But since I have a working Visual Basic decoding program, and I had a few spare hours, I took up the challenge of converting it to Visual c++, and after making every possible mistake, I got it working. I was trying to make it so it would accept inputfile and outputfile as command line parameters, but I ran into a problem with wide characters that I'm still trying to solve. My workaround is to put the file names in a file called Base64Files.txt, which can be created any old way, and has the form

Code: Select all

C:\\Documents and Settings\\Charlie\\My Documents\\inputfile.txt
C:\\Documents and Settings\\Charlie\\My Documents\\outputfile.bmp

Where inputfile must be a text file, and outputfile needs, of course to be the same file type as the input file was encoded from.

Here is the program, though it badly needs documented.

Code: Select all

// base64decode.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <fstream>
#include <iostream>
#include <string.h>
#include <conio.h>
#include <stdlib.h>
using namespace std;


class deCoded {
public:
	long len;
	char *str;
	deCoded(long,long);
};

deCoded::deCoded(long c,long l) {
	len = c;
	str = (char *) calloc(l,1);
}

deCoded Base64Decode(char *StringIn);
void ConvertBase64File(char *InputFileName, char *OutputFileName);

int _tmain(int argc, _TCHAR* argv[])
{
	char *InputFileName = (char *) calloc(1024,1);
	char *OutputFileName = (char *) calloc(1024,1);
	ifstream Base64Files;
	
	Base64Files.open("C:\\Documents and Settings\\Charlie\\My Documents\\Base64Files.txt", ios::in);
	Base64Files.getline(InputFileName,1024,'\n');
	Base64Files.getline(OutputFileName,1024, '\n');
	Base64Files.close();
	ConvertBase64File(InputFileName, OutputFileName);
	return 0;
}

deCoded Base64Decode(char *StringIn)
{
	char *sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	long bits24;
	char *right2;
	char *right1;
	deCoded decodeStr(0,4);

	right2 = &StringIn[strlen(StringIn) - 2];
	right1 = &StringIn[strlen(StringIn) - 1];
	
	if (strcmp(right2,"==") && strcmp(right1,"=")) {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion) +
				64 * (strchr(sConversion,StringIn[2]) - sConversion) + 
				(strchr(sConversion,StringIn[3]) - sConversion);		
        (decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		(decodeStr.str)[1] = (int) (bits24 & 0x0000ff00L)/256;
		(decodeStr.str)[2] = (int) bits24 & 0x000000ffL;
		
		decodeStr.len = 2;
	}
	else if (strcmp(right2,"==") && !strcmp(right1,"=")) {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion) +
				64 * (strchr(sConversion,StringIn[2]) - sConversion);
				
        (decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		(decodeStr.str)[1] = (int) (bits24 & 0x0000ff00L)/256;
		decodeStr.len = 1;
	}
	else {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion);
								
		(decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		decodeStr.len = 0;
	}
	return decodeStr;
}

void ConvertBase64File(char *InputFileName, char *OutputFileName)
{
	ofstream OutputWriter;
    ifstream InputReader;
    char *sInputLine = (char *) calloc(5,1);
	deCoded DecodeStr(0,4);
    int  i;

    OutputWriter.open(OutputFileName, ios::out | ios::trunc | ios::binary);
    InputReader.open(InputFileName, ios::in);
    InputReader >> sInputLine;
    while (InputReader.good())
	{
		DecodeStr = Base64Decode(sInputLine);
		for(i = 0;i <= DecodeStr.len;i++)
		{
			OutputWriter.write(&(DecodeStr.str)[i],1);
		}
        InputReader >> sInputLine;
	}
    InputReader.close();
    OutputWriter.close();
}

It is a Windows console application as it stands, though with some minor tweaking it should be portable across platforms and compilers. After Base64Files.txt has been set up, base64decode.exe can be run from the command line, or from within Calc by

Code: Select all

Sub VCDecodeFileFromBase64
   Dim SysShell
   Dim ProgramUrl As String
   ProgramUrl =  "C:\Documents and Settings\Charlie\my documents\visual studio 2010\Projects\base64decode\Debug\base64decode.exe"
   SysShell = createUNOService("com.sun.star.system.SystemShellExecute")
   SysShell.Execute(ProgramUrl,"",0)
End Sub

with the ProgramUrl set to the desired location, of course.

This may not be ideal solution, but it has one pretty big advantage over a pure Calc macro: since it is a compiled c++ program, it will convert even a rather large file in pretty much nothing flat.

Since I've already made every possible mistake , next I'm going to try the same trick with the encoding routines.

[zabolyx]

Wow man... not what I was expecting.. but if you get that magic going for the encoding that would make my program much faster... is there a good way of embedding that within the calc document itself?

If not I can have this be an external download from the FTP server and ran from there. Using a compiled list to execute against (created by the spreadsheet) and use it to encode the entire directory of files.. I can then reuse most of those files again... because they will be reusable as they are tied to a property in our system... Man Sir Charles.. you not only got game but you rewrote the rules to the game.

OH... just thought of something... can't C++ code be used to create code for OOo?...

[Charlie Young]

Well, I got the encode routines working in c++ (the tests so far anyway). I hadn't made every possible mistake yet of course. I spent hours with a frustrating bug, finally looked up the rules for type conversions, and discovered that I needed to be using unsigned char instead of plain char in the encoded class. c++ gurus may notice that I went to unsigned in a few places where it wasn't really necessary.

Here I'm using FilesBase64.txt to specify the input and output files, with the file to be converted as the first line and the encoded text file second. Just the reversal of Base64Files.txt from the decoding program. I would still like to get the command line arguments working.

Code: Select all

// Base64Encode.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <fstream>
#include <iostream>
#include <string.h>
#include <conio.h>
#include <stdlib.h>
using namespace std;


class encoded {
public:
	long len;
	unsigned char *str;
	encoded(long,long);
};

encoded::encoded(long c,long l) {
	len = c;
	str = (unsigned char *) calloc(l,1);
}

unsigned char *Base64Encode(encoded enCode);
void EncodeFiletoBase64(char *InputFileName, char *OutputFileName);

int _tmain(int argc, _TCHAR* argv[])
{
	char *InputFileName = (char *) calloc(1024,1);
	char *OutputFileName = (char *) calloc(1024,1);
	ifstream FilesBase64;
	
	FilesBase64.open("C:\\Documents and Settings\\Charlie\\My Documents\\FilesBase64.txt", ios::in);
	FilesBase64.getline(InputFileName,1024,'\n');
	FilesBase64.getline(OutputFileName,1024, '\n');
	FilesBase64.close();
	EncodeFiletoBase64(InputFileName, OutputFileName);
	return 0;
}

unsigned char *Base64Encode(encoded enCode)
{
	unsigned char *sConversion = (unsigned char *) "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	long Bytes3;
	long StringLength;
	unsigned char *Base64Convert = (unsigned char*) calloc(5,1);

	StringLength = enCode.len;

	if (StringLength == 3) {
		Bytes3 = 256*256*(enCode.str)[0] + 256*(enCode.str)[1] + (enCode.str)[2];
		Base64Convert[0] = sConversion[(Bytes3 & 0x00fc0000L)/(4*256*256)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x0003f000L)/(4*4*256)];
		Base64Convert[2] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[3] = sConversion[(Bytes3 & 0x0000003fL)];
	}
	else if (StringLength == 2) {
		Bytes3 = 256*(enCode.str)[0] + (enCode.str)[1];
		Bytes3 <<= 2;
		Base64Convert[0] = sConversion[(Bytes3 & 0x0003f000L)/(4*4*256)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[2] = sConversion[(Bytes3 & 0x0000003fL)];
		Base64Convert[3] = (unsigned char) '=';
	} else {
		Bytes3 = (enCode.str)[0];
		Bytes3 <<= 4;
		Base64Convert[0] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x0000003fL)];
		Base64Convert[2] = (unsigned char) '=';
		Base64Convert[3] = (unsigned char) '=';
	}
	return Base64Convert;
}

void EncodeFiletoBase64(char *InputFileName, char *OutputFileName)
{
	ofstream OutputWriter;
	ofstream GraphicWriter;
    ifstream InputReader;
    //char *OutputLine = (char *) calloc(5,1);
	encoded enCode(0,4);
	long i;
	char inByte;
	
    InputReader.open(InputFileName, ios::in | ios::binary);
    OutputWriter.open(OutputFileName, ios::out | ios::trunc);
	
    i = 0;
	InputReader.read(&inByte,1);
	while (!InputReader.eof())
	{
		(enCode.str)[i] = (unsigned char) inByte;
		++i %= 3;
		if (i == 0) {
			enCode.len = 3;
			OutputWriter << Base64Encode(enCode) << endl;
		}
		InputReader.read(&inByte,1);
	}
	if (i > 0) {
		enCode.len = i;
		OutputWriter << Base64Encode(enCode) << endl;
	}
	InputReader.close();
    OutputWriter.close();
}

As far as integrating this more tightly into OOo, I think one needs to become familiar with the SDK, which I just finally got around to downloading, and I have a lot to learn...

[zabolyx]

One thing I do notice with this is that the location of the txt file to read is a constant in this program... should that be setup as the same directory of as the application or be passable to the program as a parameter?

This is great Charlie... I think I can have my spreadsheet build a list of the files that need converted and convert those for the user and then use the shell commands to append the created file onto the end of the eml file... open and write what is needed to the file and append the next segment... I think this would be faster than to open the needed files and copy the data from one to the next. Windows should handle that fairly easily to help improve speeds to an astronomical rate.

What about a compiled version of this wizardry..... I don't happen to have VC++ or have any competence in using a C++ compiler... Still on my list of languages to learn as C++ is too versatile to ignore.

[Charlie Young]

What about a compiled version of this wizardry..... I don't happen to have VC++ or have any competence in using a C++ compiler... Still on my list of languages to learn as C++ is too versatile to ignore.

Visual c++ Express is free:

http://www.microsoft.com/express/Windows/

You just need to register to go beyond the trial period.

It is not allowed to attach .exe files here, so I zipped both the encode and decode executables. They were compiled under XP, so hopefully they'll work for you. I want to try the .cpp files with the old Borland compiler I still have to see if I can port them that way.

I suppose there also may be an issue of licensing with Microsoft if you use this for commercial use, so I'm not suggesting you do so.

[Charlie Young]

One thing I do notice with this is that the location of the txt file to read is a constant in this program... should that be setup as the same directory of as the application or be passable to the program as a parameter?

Duh. I just realized that this will make the use of the attached executables problematic. I now have the command line problem solved though (I guess). I'll attach new versions shortly.

[Charlie Young]

All right, my find is totally fried by now and it's past my bedtime, but here goes.

I have modified the encode and decode programs to work with command line parameters. As I indicated above, the was a problem with VC++ reading these parameters (argv[] passed to _tmain) because they are passed in as unicode characters. I had recompiled the programs using the Borland compiler after making some small changes, simply changed

#include "stdafx.h"

to

#include <stdio.h>

and changed

int _tmain(int argc, _TCHAR* argv[])

to the old familiar

int main(int argc, char *argv[])

and found that the recompiled version read the command line arguments without further ado. But then I went back to VC++, googled a bit and found out how to read the unicode argv[], but since I really didn't need the unicode chars anywhere else in the program, I wrote my own little conversion function which works by just picking off the first byte of the two byte unicode chars.

Code: Select all

char* wchar2char(_TCHAR *s)
{
	char *c = (char *) calloc(1024, 1);	
	long i, l;

	l = wcslen(s);
	for (i = 0;i < l;i ++)
		c[i] = ((char *) s)[2*i];
	return c;
}

With that in place, the code for the encode program is

Code: Select all

// clBase64Encode.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <fstream>
#include <io.h>
#include <iostream>
#include <string.h>
#include <conio.h>
#include <stdlib.h>


using namespace std;


class encoded {
public:
	long len;
	unsigned char *str;
	encoded(long,long);
};

encoded::encoded(long c,long l) {
	len = c;
	str = (unsigned char *) calloc(l,1);
}

unsigned char *Base64Encode(encoded enCode);
void EncodeFiletoBase64(char *InputFileName, char *OutputFileName);
char* wchar2char(_TCHAR *s);

int _tmain(int argc, _TCHAR* argv[])
{
	char *InputFileName = (char *) calloc(2048,1);
	char *OutputFileName = (char *) calloc(2048,1);
	
	if (argc == 3) {
		strcpy_s(InputFileName, 1024, wchar2char(argv[1]));
		strcpy_s(OutputFileName, 1024, wchar2char(argv[2]));
	}
	else {
		cout << "Incorrect number of parameters." << endl;
		cout << "Usage: clBase64Encode Inputfile Outputfile" << endl <<  endl;
		cout << "Hit any key to exit.";
		_getch();
		exit(1);
	}
	
	EncodeFiletoBase64(InputFileName, OutputFileName);
	return 0;
}


unsigned char *Base64Encode(encoded enCode)
{
	unsigned char *sConversion = (unsigned char *) "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	long Bytes3;
	long StringLength;
	unsigned char *Base64Convert = (unsigned char*) calloc(5,1);

	StringLength = enCode.len;

	if (StringLength == 3) {
		Bytes3 = 256*256*(enCode.str)[0] + 256*(enCode.str)[1] + (enCode.str)[2];
		Base64Convert[0] = sConversion[(Bytes3 & 0x00fc0000L)/(4*256*256)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x0003f000L)/(4*4*256)];
		Base64Convert[2] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[3] = sConversion[(Bytes3 & 0x0000003fL)];
	}
	else if (StringLength == 2) {
		Bytes3 = 256*(enCode.str)[0] + (enCode.str)[1];
		Bytes3 <<= 2;
		Base64Convert[0] = sConversion[(Bytes3 & 0x0003f000L)/(4*4*256)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[2] = sConversion[(Bytes3 & 0x0000003fL)];
		Base64Convert[3] = (unsigned char) '=';
	} 
	else {
		Bytes3 = (enCode.str)[0];
		Bytes3 <<= 4;
		Base64Convert[0] = sConversion[(Bytes3 & 0x00000fc0L)/(4*4*4)];
		Base64Convert[1] = sConversion[(Bytes3 & 0x0000003fL)];
		Base64Convert[2] = (unsigned char) '=';
		Base64Convert[3] = (unsigned char) '=';
	}
	return Base64Convert;
}

void EncodeFiletoBase64(char *InputFileName, char *OutputFileName)
{
	ofstream OutputWriter;
	ifstream InputReader;
    encoded enCode(0,4);
	long i;
	char inByte;
	
	if(_access(InputFileName, 0) == -1 ) {
		cout << "Input file does not exist."  << endl << endl;
		cout << "*** Hit any key to exit. ***";
		_getch();
		exit(1);
	}																					
	
    InputReader.open(InputFileName, ios::in | ios::binary);

    OutputWriter.open(OutputFileName, ios::out | ios::trunc);
	
    i = 0;
	InputReader.read(&inByte,1);
	while (!InputReader.eof())
	{
		(enCode.str)[i] = (unsigned char) inByte;
		++i %= 3;
		if (i == 0) {
			enCode.len = 3;
			OutputWriter << Base64Encode(enCode) << endl;
		}
		InputReader.read(&inByte,1);
	}
	if (i > 0) {
		enCode.len = i;
		OutputWriter << Base64Encode(enCode) << endl;
	}
	InputReader.close();
    OutputWriter.close();
}

char* wchar2char(_TCHAR *s)
{
	char *c = (char *) calloc(1024, 1);	
	long i, l;

	l = wcslen(s);
	for (i = 0;i < l;i ++)
		c[i] = ((char *) s)[2*i];
	return c;
}

And the decode program

Code: Select all

// clBase64Decode.cpp : Defines the entry point for the console application.
//


#include "stdafx.h"
#include <fstream>
#include <iostream>
#include <io.h>
#include <string.h>
#include <conio.h>
#include <stdlib.h>
using namespace std;


class deCoded {
public:
	long len;
	char *str;
	deCoded(long,long);
};

deCoded::deCoded(long c,long l) {
	len = c;
	str = (char *) calloc(l,1);
}

deCoded Base64Decode(char *StringIn);
void ConvertBase64File(char *InputFileName, char *OutputFileName);
char* wchar2char(_TCHAR *s);

int _tmain(int argc, _TCHAR* argv[])
{
	char *InputFileName = (char *) calloc(2048,1);
	char *OutputFileName = (char *) calloc(2048,1);

	if (argc == 3) {
		strcpy_s(InputFileName, 1024, wchar2char(argv[1]));
		strcpy_s(OutputFileName, 1024, wchar2char(argv[2]));
	}
	else {
		cout << "Incorrect number of parameters." << endl;
		cout << "Usage: clBase64Encode Inputfile Outputfile" << endl <<  endl;
		cout << "Hit any key to exit.";
		_getch();
		exit(1);
	}
	
	ConvertBase64File(InputFileName, OutputFileName);
	return 0;
}

deCoded Base64Decode(char *StringIn)
{
	char *sConversion = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	long bits24;
	char *right2;
	char *right1;
	deCoded decodeStr(0,4);

	right2 = &StringIn[strlen(StringIn) - 2];
	right1 = &StringIn[strlen(StringIn) - 1];
	
	if (strcmp(right2,"==") && strcmp(right1,"=")) {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion) +
				64 * (strchr(sConversion,StringIn[2]) - sConversion) + 
				(strchr(sConversion,StringIn[3]) - sConversion);		
        (decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		(decodeStr.str)[1] = (int) (bits24 & 0x0000ff00L)/256;
		(decodeStr.str)[2] = (int) bits24 & 0x000000ffL;
		
		decodeStr.len = 2;
	}
	else if (strcmp(right2,"==") && !strcmp(right1,"=")) {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion) +
				64 * (strchr(sConversion,StringIn[2]) - sConversion);
				
        (decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		(decodeStr.str)[1] = (int) (bits24 & 0x0000ff00L)/256;
		decodeStr.len = 1;
	}
	else {
		bits24 = 64 * 64 * 64 * (strchr(sConversion,StringIn[0]) - sConversion) + 
				64 * 64 * (strchr(sConversion,StringIn[1]) - sConversion);
								
		(decodeStr.str)[0] = (int) (bits24 & 0x00ff0000L)/(256 * 256);
		decodeStr.len = 0;
	}
	return decodeStr;
}

void ConvertBase64File(char *InputFileName, char *OutputFileName)
{
	ofstream OutputWriter;
    ifstream InputReader;
    char *sInputLine = (char *) calloc(5,1);
	deCoded DecodeStr(0,4);
    int  i;

	if(_access(InputFileName, 0) == -1 ) {
		cout << "Input file does not exist."  << endl << endl;
		cout << "*** Hit any key to exit. ***";
		_getch();
		exit(1);
	}
    OutputWriter.open(OutputFileName, ios::out | ios::trunc | ios::binary);
    InputReader.open(InputFileName, ios::in);
    InputReader >> sInputLine;
    while (InputReader.good())
	{
		DecodeStr = Base64Decode(sInputLine);
		for(i = 0;i <= DecodeStr.len;i++)
		{
			OutputWriter.write(&(DecodeStr.str)[i],1);
		}
        InputReader >> sInputLine;
	}
    InputReader.close();
    OutputWriter.close();
}

char* wchar2char(_TCHAR *s)
{
	char *c = (char *) calloc(1024, 1);	
	long i, l;

	l = wcslen(s);
	for (i = 0;i < l;i ++)
		c[i] = ((char *) s)[2*i];
	return c;
}

the usage for these is

clBase64Encode InputFile OutputFile

and

clBase64Decode InputFile OutputFile

where the file names need to be quoted if there are any blanks in the path and/or file name, and that in good c fashion, the backslash characters need to be doubled, e.g.

"C:\\Documents and Settings\\Charlie\\My Documents\\inputfile.txt"

(recall that the output file for encoding and the input file for decoding need to be text files.)

Lastly (for now), to run these in a shell from a Calc macro, do something like

Code: Select all

Sub VCDecodeFileFromBase64(InputFileName As String, OutputFileName As String)
   Dim SysShell
   Dim ProgramUrl As String
   Dim CommandLine As String
   
   'Set ProgramUrl to local setup, exe can be anywhere.
   ProgramUrl = "C:\Documents and Settings\Charlie\my documents\visual studio 2010\Projects\clbase64decode\Debug\clBase64decode.exe"
   CommandLine = InputFileName & " " & OutputFileName
   SysShell = createUNOService("com.sun.star.system.SystemShellExecute")
   SysShell.Execute(ProgramUrl,CommandLine,0)
End Sub

and correspondingly for encode.

Note that here, to quote the file names, use Chr(34) or triple the " characters

InputFileName = """C:\\Documents and Settings\\Charlie\\My Documents\\inputfile.txt"""

The new zipped executables are attached.

[zabolyx]

I am having an issue with using these... Seems that they need a dll... maybe the 2010 VC++ redistributable package... which I can install on the end users machines... I'm downloading it now to get that a try

Not doing it for me... still getting error needing MSVCP100.dll

[Charlie Young]

I am having an issue with using these... Seems that they need a dll... maybe the 2010 VC++ redistributable package... which I can install on the end users machines... I'm downloading it now to get that a try

Not doing it for me... still getting error needing MSVCP100.dll

I was afraid there would be some portability problems. The redistributable is supposed to fix those, but I guess it doesn't do the whole job

On my system MSVCP100.dll is in C:\WINDOWS\SYSTEM32. Google indicates a few places it can be downloaded from, it's too big to attach here.

It's a nuisance that any of this is necessary with what should basically be a DOS program.

[Charlie Young]

I changed some compiler options. See if the newly attached versions work better.

[rudolfo]

Okay, I know it is not the best style to jump into a long discussion with a lot of coding hours in the background. But if you compile a standalone commandline application for doing the encoding/decoding this is like saying "I have reached the limits what Basic can do for me".
At this point it might be a good idea to look for other scripting languages. I have python in mind because it comes with a base64 module in its standard library. In other words it is included with the python interpreter that comes with OpenOffice. Unfortunately I have not enough ensight into python, but hopefully they implemented the core of this module in C for the sake of speed. Yes, just had a quick look, base64 uses binascii and for this module the documentation says: "The binascii module contains low-level functions written in C for greater speed that are used by the higher-level modules."

As you are working on a Win32 platform I guess the better performance in the C++ Code will not always compensate for the performance penalty that you get when starting a new process for the commandline executable. Right, for large files (>30 kByte) you are surely on the positive side.
Alternatively you might compile the base64 algorithm into a DLL and use it with "Declare ..." to use a exported Dll function from Basic. (See the Declare Statement in the Online Help of OOo Basic). This would save you the overhead of starting a new process for every file that will be encoded.

[zabolyx]

Rudolfo... You're right and OOo is the perfect playground to learn a number of new languages... And I would love to do that in the future. But sometimes a separate module is needed for data intensive coding for any language (as long as it can be done faster in another). For example using ASM to write code to use in C++. At the moment this is what I can use... so I shall.

At this point in time I'm just needing to get the work done and I do know BASIC... well I have experience using BASIC. Not use that it's really knowledge.

Python and Java are on my list of languages to learn... and once this project is done and slimmed down as much as I can get it I plan on rewriting it in Java... but who knows I might try some Python first. Which has to wait until I have SQL under the belt and JavaScript started on (these are needed to implement a few things I need to work on for my job).

As for style... any input, as long as it is contributing to the discussion, is always acceptable.

I'm not sure how much of a penalty we get from the file that executes and terminates once done. I also have a loop in the code that checks if the new file has been created before continuing... thus keeping more than a single instance of the program from running. If I just looped through as fast as BASIC could it would be possible to have the program running dozens of copies.

Now I must say that the Declared DLL sounds pretty sexy... I'd love to work on that concept in the future... it would be very nice to have a collection of DLL files that could be directly accessed from BASIC for use (or other languages).

Charles my man... these seem to be working beautifully. I'll give them a few tests and see how we come out. I'm needing to get my newest BETA release done for testing by my cube-mate.