Re: Receive adapter deletes empty files, doesn't forward on message!

Tech-Archive recommends: Fix windows errors by optimizing your registry

From: Xerox (info_at_thinkscape.com)
Date: 02/16/05


Date: Wed, 16 Feb 2022 19:54:54 -0000

Thanks for your input and suggestions Nabeel - they are very useful.
Unfortunately for us, we deal a lot with legacy systems that use zero byte
files that have important meaning, so we must find a way forwards if we are
to use BizTalk. I have some experience with adapter development and so do
not find the challenge insurmountable - but I do know that the company would
prefer not to have custom code written if it can avoid it, particularly for
something as fundamental as file and FTP transfer.

I would hope that Microsoft update the FTP and file adapter in the way that
you suggest; that is, to provide a flag on the adapter to indicate whether
zero byte files should be ignored or passed on. But as is always with these
things, their time scales are unlikely to coincide favourably with ours.

The workarounds on the receive side are to either use the SDK file adapter,
which seems to work well (but it uses polling to check for new files which
seems a backward step compared to the event mechanism that the out of the
box file adapter uses). Or to use the FTP receive adapter, again as you
suggest.

The send adapter side of things is looking less promising. There is no FTP
send adapter in the SDK nor can we use the file send adapter - FTP is the
only option. Therefore using an orchestration or developing a custom FTP
send adapter are the only options open to us.

However, option 3. sounds intriguing - if it works it could be great! I'll
give it a try.

"Nabeel Prior" <nowhere@somewhere.com> wrote in message
news:OoZdydFFFHA.2540@TK2MSFTNGP09.phx.gbl...
> Xerox,
>
> Sorry for the delayed response.
>
> With respect to the first question, regarding you to submit as a change
> request, I am not sure, and would try and make an appeal to someone like
> Scott Woodgate as to the identify the correct process to follow.
>
> With respect to your subsequent findings on how BizTalk process empty
files:
> >> The FILE receive adapter rejects 0 byte files
> >> The FTP receive adapter accepts 0 byte files
>
> This is inconsistent, and should be remedied. I can understand both sides
> of the logic: one side would say that you should not process an empty file
> because the file itself is empty, while another side would say I want to
be
> able to process the file because there is some significance in the way the
> file has been named (e.g. sequence tagging). My feeling here is that
there
> should therefore be an ability to receive empty files, with a flag to
either
> pass them on or not. This would then allow us to utilise the pipeline
> components to act on the receipt of an empty file, if we choose to.
Lastly,
> this is not so important with XML files as one would never expect them to
be
> empty. But for flat files, especially those generated by legacy systems,
> there is often reasoning built into the naming of the files, and for this
> reason alone we should be able to process empty files.
>
> >> The FILE send adapter sends 0 byte files
> >> The FTP send adapter rejects 0 byte files
>
> Although this is consistent, for the same reason as above you should have
> the option to send empty files because of the potential importance to the
> destination system of the file's name.
>
> Solution: There is no elegant way to address this, keeping in mind that
> BizTalk's message box expects an XML message to facilitate routing, and an
> empty file is not XML. Short of waiting on a potential change request
being
> implemented from MS, I would suggest a couple of possibilities:
>
> Receiving an empty file:
> 1. If possible, get all the receives to occur via FTP, so that you can at
> least get the message into the pipeline. Once in the pipeline, use a
> probing pipeline component to identify that the message is empty and
> generate a new XML message that contains the file name, which can then be
> sent to the message box and handled as a specific message type.
> 2. Create your own FILE / FTP adapter that will receive messages and pass
> them on (as the existing one does), but if the message is empty, it will
> create a new XML message containing the file name for further processing.
> You may be able to implement this as a wrapper on the existing file
adapter.
>
> Sending an empty file:
> 1. Create your own FILE / FTP send adapter that will be able to handle a
> zero-byte file. Once again, you may be able to wrap the existing adapters
> with your own logic.
> 2. Utilise a .Net assembly from within the orchestration to write the file
> out / send the file via FTP.
> 3. Not sure if this is possible, but perhaps using a dynamic FTP port in
an
> orchestration you could utilise the Before PUT command to instruct the
> creation of a file on the FTP server, if the file is empty, otherwise you
> could send the file that contains data.
>
> Just some thoughts that you can perhaps work with?
>
> --
>
> Kind Regards,
>
> Nabeel Prior
> Web: http://www.brainjar.co.za
> Blog: http://www.brainjar.co.za/blog.aspx
>
>
> "Xerox" <anon@anon.com> wrote in message
> news:%23$N6MHCFFHA.2452@TK2MSFTNGP09.phx.gbl...
> >I am crying now. The flat file disassembler fails on zero byte files. So
it
> > doesn't even matter if I can read them - it screws up later on in the
> > process.
> >
> > This gives us only one option - and that is to make changes to all the
> > systems that send us files and to all the systems that we send files to.
> > That's the complete opposite of what an EAI product is supposed to do -
> > its
> > supposed to help integrate systems, not force those systems to change
> > because of it!
> >
> > Agh!
> >
> >
> > "Xerox" <anon@anon.com> wrote in message
> > news:uqWeb6BFFHA.1836@tk2msftngp13.phx.gbl...
> >> I am going to cry - I don't believe it... the FTP send adapter doesn't
> >> transfer files with 0 bytes... but the FILE send adapter does!!! This
is
> >> nuts.
> >>
> >> So:
> >> The FILE receive adapter rejects 0 byte files
> >> The FTP receive adapter accepts 0 byte files
> >> The FILE send adapter sends 0 byte files
> >> The FTP send adapter rejects 0 byte files
> >>
> >> How can this be sensible? Has anyone else run in to these issues?
> >>
> >> This is making BizTalk look like in such a bad light here. How can we
> >> work
> >> with a huge inconsistency such as this?
> >>
> >> "Xerox" <anon@anon.com> wrote in message
> >> news:uAAH9W0EFHA.3928@TK2MSFTNGP15.phx.gbl...
> >> > I have encountered a big issue with the file receive adapters: they
> > delete
> >> > empty messages and don't pass them on to BizTalk. This is a real
> >> > problem
> >> for
> >> > us as our system expects files to be received in sequence. Empty
files
> > are
> >> > perfectly valid. If BizTalk is deleting them then our system just
> >> > stops,
> >> > waiting forever for a file that will never be received.
> >> >
> >> > What can we do?
> >> >
> >> >
> >>
> >>
> >
> >
>
>



Relevant Pages

  • Re: Receive adapter deletes empty files, doesnt forward on message!
    ... With respect to your subsequent findings on how BizTalk process empty files: ... >> The FILE receive adapter rejects 0 byte files ... >> The FTP receive adapter accepts 0 byte files ... generate a new XML message that contains the file name, ...
    (microsoft.public.biztalk.general)
  • FTP Receive Location Failures
    ... I'm seeing some unexpected behavior using the FTP Receive Adapter. ... I am receiving flat-file purchase orders using the FTP Receive Adapter and ... document to the custom pipeline and if the document fails validation the ...
    (microsoft.public.biztalk.general)
  • Re: FTP Adapter issue
    ... Looks like the FTP protocol on the server is not compliant with the ... supported by MS for FTP Server and mine one is not there its HP OpenVMS. ... If the adapter does not see any files matching the file mask after ...
    (microsoft.public.biztalk.general)
  • Biztalk 2004: FTP Receive Adapter - Not Deleting Files Consistantly
    ... Since I had to switch to using this from the File receive adapter based ... on issues we were having with that adapter, I have noticed that the FTP ... the folder, according to the log, the FTP adapter: ... submitted a garbage file to a pipeline with a flat file disassembler. ...
    (microsoft.public.biztalk.general)
  • Biztalk 2004: FTP Receive Adapter - Not Deleting Files Consistantly
    ... Since I had to switch to using this from the File receive adapter based ... on issues we were having with that adapter, I have noticed that the FTP ... the folder, according to the log, the FTP adapter: ... submitted a garbage file to a pipeline with a flat file disassembler. ...
    (microsoft.public.biztalk.general)