Draining the happy soup - Part 1
Unleashing unlocked packages promises to reduce risk, improve agility and drive home the full benefits of SFDX
Some planning required
I'm following the approach "Throw and see what sticks to the wall". The rough idea: retrieve all meta data, convert it into SFDX format, distribute it over a number of packages and put it back together.
To make it more fun I picked an heavily ~~abused~~ customized and used org with more than 20,000 meta data artifacts (and a few surprises). Follow along.
Learning
Trailhead has a module on unlocked packages on its trail Get Started with Salesforce DX.
While you are there, check out the (at time of writing the 15) modules on Application Lifecycle Management.
Downloading
The limits for retrieving packages (10,000 elements, 39MB zip or about 400 MB raw) posed an issue for my XL org. So I used, growing fond of it, PackageBuilder to download all sources. It automatically creates multiple package.xml files when you exceed the limits.
Of course, no action without headache. I ran out of memory on my first attempts, so I had to tweak the jvm parameters with -Xms512M -Xmx1G
. Furthermore I limited the number of items to retrieve to 4000 per package and excluded the meta data type Document
. Documents, not visible in Lightning are user creatable, why they are meta data is beyond me. For reference my build.properties
file:
# Package builder sf.apiversion=45.0 sf.username=user@salesforce.io sf.password=Sup3rS3cr3t sf.serverurl=https://test.salesforce.com sf.maxPoll=200 includechangedata=false metadatatargetdir=src destination=packages download=true unpack=false gitcommit=false skipItems=Document:* maxitems=4000
You might need to play with your own parameters. A little experimenting goes a long way.
Conversion
You create a SFDX project sfdx force:project:create
and you have moved the zip files to the root of the project. Now unzip one archive to mdapisrc
and execute
sfdx force:mdapi:convert -r mdapisrc -d force-app
. Rinse and repeat.
So far so good. Most likely, until I get it right, I have to repeat these steps more than once, so I created a small shell script (time to get a Mac or the Windows Subsystem for Linux) to repeat the whole exercise:
#!/bin/bash TITLE="Happy Soup to SFDX retrieval" SECONDS=0 process_package() { rm -rf mdapisrc/* unzip -d mdapisrc $1.zip sfdx force:mdapi:convert -r mdapisrc -d force-app } #Create packages java -Xms512M -Xmx1G -jar ~/bin/PackageBuilder.jar -o build.properties -download # Retrive the packages and convert # First the one without a number process_package package # Then the numbered ones for fname in packages/package.*.xml; do fn="${fname%.xml}" f=${fn##*/} process_package $f done #Time information LAPSED="$(($SECONDS / 3600))hrs $((($SECONDS / 60) % 60))min $(($SECONDS % 60))sec" MESSAGE="SFDX Operations completed in: $LAPSED" #Let the Mac know osascript -e "display notification \"${MESSAGE}\" with title \"${TITLE}\"" echo "Done with $TITLE $MESSAGE"
On Linux you want to replace the osascript
call with growl
or notify-send
or skip it completely.
Now we got local files to play with, they need to be distributed to their new projects and kept current. But that's the story for next time.
As usual YMMV
Posted by Stephan H Wissel on 14 February 2019 | Comments (0) | categories: Salesforce SFDX