Glue Development

Following are the steps to deploy artifacts manually to development environment:

1. Go to wap-audit-transform root.
2. Run mvn clean install, make sure the build passes.
3. Upload the build output. Following are the files to be uploaded to wap-audit-shipment bucket and the folders where they should be uploaded (note that you should replace the name of the version x.y.z according to the version that you’re uploading):

   File	Target Folder
   wap-audit-transform-glue/src/main/scala/AuditGlueTransformApp.scala	etl/wap-audit-transform-glue/x.y.z/
   wap-audit-transform-hue/target/hue-x.y.z-dependencies.jar	etl/wap-audit-transform-hue/x.y.z/
   wap-audit-transform-integrated/target/integrated-x.y.z-dependencies.jar	etl/wap-audit-transform-integrated/x.y.z/

4. Update job definition to use the new uploaded files:
   1. For HUE job, edit wap-audit-etl-hue-{env}, set “Script path” field to s3://wap-audit-shipment/etl/wap-audit-transform-glue/x.y.z/AuditGlueTransformApp.scala
   2. Set “Dependent jars path” field to s3://wap-audit-shipment/etl/wap-audit-transform-hue/x.y.z/hue-x.y.z-dependencies.jar
   3. Press “Save” button.