Complex flat file stage datastage example programs

broken image
broken image
broken image

Merge combines one master input with multiple update inputs according to values of a key column(s).Records don’t need to be sorted and produces single output stream and a reject link. Lookup stage can have 1 source and multiple lookup tables. Lookup combines two or more inputs according to values of a key column(s).Can have 1 left and multiple right inputs (all need to be sorted) and produces single output stream (no reject link). Similar concept to relational DBMS SQL join (ability to perform inner, left, right and full outer joins). Join combines two or more inputs according to values of a key column(s).Funnel combines multiple streams into one.Filter filters out records that do not meet specified requirements.FTP stage uses FTP protocol to transfer data to a remote machine.Copy – copies input data (a single stream) to one or more output data flows.The data can be grouped using two methods: hash table or pre-sort. Aggregator joins data vertically by grouping incoming data stream and calculating summaries (sum, count, min, max, variance, etc.) for each group.