Similar to the PDI bulk load LAN method create a WAN method to retrieve source data via AppServer procedure calls.

The following are the specs and answers to questions I received for direction:

While we may not be able to pull off ALL of this, here are the basic items that bulk load does (WAN or LAN):
1. Looks up replControl status. If completed, then quit, if “in process” then find last rows (using cache) and continue.
2. Keeps “cache” of the last X rowids processed. There MIGHT be a replProperty of name BULK_MAX_CACHE. If it exists we use that number when keeping cache. Otherwise we default to 25.
3. Every 100k, writes status to a log.
4. Every 100k, updates replControl status
ASSIGN

{&ReplDB}ReplControl.GroupID = "BULKCOPY"
{&ReplDB}

ReplControl.CodeID = "<<SOURCE_DB>>.<<SRC_TABLE>>"

{&ReplDB}ReplControl.CodeVal1 = "IN PROCESS"
{&ReplDB}

ReplControl.CodeVal2 = STRING(TODAY) + "-" + STRING(TIME,"HH:MM:SS")

{&ReplDB}

ReplControl.CodeVal3 = "Count=" + STRING(intRecordNum) + ";RowidCache=" + strRowidCache.

For WAN we do:
1. Lookup APPSRV_FETCH_COUNT_BULK replProperty. If available we use this value, otherwise 1000.
2. Run Appserver “get” program (strASmProcDir + "/get_<<SOURCE_DB>>_<<SRC_TABLE>>.p") and request a “batch” of rows. The signature is:
INPUT INTEGER(cMassFetchLimit),
INPUT strRowidCache,
INPUT TABLE tt_RQ, /* 75110 */
OUTPUT TABLE bfrSrc<<SRC_TABLE25>>,
OUTPUT strStatus
3. Loop through multi-iterations of appserver calls, doing the 100k stuff as appropriate.

Q&A:
Do you think we should limit the number of errors by table? For example some issue is causing every row of a table to error so should they all be writing to ReplQueue or stop table load after X number of errors? I like this idea. We should probably make it property driven? If the property is not there, then don’t stop (currently functionality), if it is there and is a number and errors exceed the number then bail on that table?

How should higher level errors be handled like unable to connect to target database for table? Write the error to the table log and bail.

When inserting into ReplQueue I won’t have access to DBTASKID from SQL so what should I put in the SrcTransID? set it as 0