You need:
Creation Kit
Nifskope[github.com]
Blender[www.blender.org]
Outfit Studio[www.nexusmods.com]
BSA Browser[www.nexusmods.com]
Basic understanding of how to use Blender.

First of all, get to the point where you know for sure you will not expand the building area with more buildings, so you won't need to create a new border. This process is tedious and we don't want to redo it again.

Save the Mod.esp and make a copy of it, as a backup.
Because we'll most likely need to touch models that are in "precombines". Accidentally nudging them will break the precombines, which in return may affect FPS in some areas.

!!! AFTER THIS SAVE YOU MUST NOT SAVE IT ANYMORE!!!
Else you will mess up precombines and save over it.
We will need to do some temporary stuff that must not be part of your mod.

We will make a fake border wall from props, which we will then turn into a static collection.
This collection will then be exported as a NIF, imported into blender and used as a guide to make this border wall of ours.

Search for the Static model "HitExtAFloorCornerA02".
This is a really large rectangle. Place it into the world and set its scale to 10.
Turn it onto its edge and build the outline of your border.
Tip: Press T to enter "Top Down Camera" mode, so you'll get a nice view of your cell.
Follow along your "Build Area Triggers" to be precise. I did not do that, but you should.


Make platforms on the floor plane, so we know where the bottom of this border should be.
We can't export terrain. This is the best we can do.

The Top platform is a guide for the water level, so i know from where the ground actually starts.
The bottom one is the lowest point underwater, so I know how deep down I must go.


Select all the wall pieces that you had just placed, along with any other items you want as a reference.

Then at the end, select one model that you want to center your borders on.
It must be any item that is part of the settlement. Some static model near the workbench will do.
I chose the roof of the terminal. Note how the movement gizmo is on the roof.


While selection is active, right click in the Render Window and click "Make Static Collection.


It'll ask you for a name. Pick something easy. i just do 00_Wshop_Border.
But you could also just do 000000000000, doesn't matter. it's temporary


It will ask you again for something in an identical text box. Leave it empty, just click on "OK"

Look for the static collection you just made in the Object Window.


Right click it and click "edit".
In the new window, click on "Recreate NIF from Data".


Close the creation kit. It will ask you if you want to save, Click "No". DO NOT SAVE!
We'll continue with NifSkope and your modeling software of choice.

Go to "Fallout 4\Data\meshes\SCOL\<modname>.esp".
You'll find a NIF file, which is your static collection of the guide you made.


Open it in NIFskope and export it as an OBJ


After doing this, you could delete the NIF file. it'll be useless now.

Export an existing workshop border from the game's BSA using BSA Browser.
We'll need it for the material setup later on.


Import your guide OBJ into blender.


The small pink thing is Red Rocket's border.
I imported it as a reference of how tall the wall should be above ground.
The default walls are 255 units tall.
But as you can see, the center of my helper model is right under that roof i had selected.


Now you can just go and start building a basic wall with the shape and size you want, using the guide.

I did some "precise" combining of meshes and whatever. But you could just take a single vertex, extrude it all around the edges of the mesh closely, form one loop and extrude the entire thing 255 units upwards.



You may need to rotate this thing -90° on the X axis.
You could also adjust it in NifSkope, but its nicer to do it now.
That's just how the Bodyslide importer imports it. IDK why.


Make sure that all normals on your border point outwards.
Name this object, which is only your border, to "B_out"
Duplicate it, invert all normals by pressing Alt+N and rename that object "B_In"
The border has green color inside and red color outside. We need both sides as separate meshes so we can color them correctly.



Export your border models "B_out" and "B_In" as two meshes in one FBX.
Select only the B_In and B_out in blender, export as FBX and click "Selected Objects"



This makes coloring them easier, because we need to manually paint the color on it.
If he hide the outside mesh, we can paint the inside all green, and then the outside all red.

Open Outfit Studio and drag your FBX file onto it.
Outfit Studio will ask to "import without Reference". Just hit okay, we don't care.
We just want the fbx to NIF conversion.

You'll now have two objects in here.


Once imported, take a looksie at it. Then Export as NIF.
I'll export it next to the FBX files i exported from blender.
We won't be putting it into the game just yet.
It'll complain about unweighted vertices and something about a mask. We don't care, click okay.


Once exported, open both your NIF and the original exported border NIF in two separate instances of NIFSkope.


Right click the original border and click on "Block - Copy Branch"


Right click your own border and click "Block - Paste Branch"


You will now have the original border, with its working material setup, inside of your NIF.


You can close the instance of the original border. We won't be needing it, or its NIF anymore.

Your models will probably be pink, while the original border works flawlessly. We need to change that real quick.

Expand the first "BSSubIndexTrishape" and delete the BSSkin and BSLighting brenches entirely.
Do the same for the second "BSSubIndexTrishape"


If done properly, your border meshes should be grey now.
But the original should still have its colors.

Click on the original border.
In the "Block Details" you need to look for the Shader Property and Alpha Property boxes.
Check what numbers are written in there.

For me, Shader is 4, Alpha is 8. For you it may be different numbers.


Now click on your meshes and enter the same numbers in those boxes.
Do that for both meshes.
They should turn from grey to white.


Click this little flag icon by "Vertex Desc" on both your meshes.
A new window will pop up, in which you need to check these boxes.
Usually is just switching from "Skinned" to "Colors"


Now you can remove the original border shape that you had used to take the materials from.
ONLY the BSTriShape!


You'll be left with two meshes that are white.


Save this NIF file. You can Name this anything you want, But i'd choose "WorkshopBorder_Modname.nif"

We need to paint the color in Outfit Studio. But Outfit's Studio's camera movement is Slow.
One trick to not needing to scroll for five minutes is to make your mesh super duper tiny.
Select the shapes and set "Scale" to 0.1, or anything small enough for the mesh to fit inside the floor grid.
0.01 is even better

Save this tiny mesh.
When you open it next time the scale value is still 0.1, we can change it back to 1 anytime.

Open Outfit Studio once more and drag your NIF onto it.
First, hide the B_IN mesh by clicking this eye, or clicking the name of the mesh and pressing E.


Select the B_OUT mesh.
Head over to color.
Set the brush color to red.
Change the Brush settings to strong as hell.


Now when you hover your mouse over the Vertices of your mesh, a green dot will appear.
Start painting the B_Out bottom vertices red.
The top ones will be black to paint the fading gradient onto it.
You could also enter Wireframe mode by clicking the Eye icon (or pressing E) twice.
Also, if the camera is too far away, it will not paint. So you'll have to zoom in.

Painting is a bit Akward, to be honest.
You gotta click and drag, sometimes wave the mouse about to get it to paint.
Here's a sped up gif of how it should look.
Just zoom real close, then it won't be hard. I learned that when painting the other half



Note that the top is white. that's because the large part on the bottom is all the parts underwater that is full red.
The top that is white will then be painted black.

Eventually, all that must be red, will be red.


Once that is done, change the brush color to black and paint the top halves black.
It will seemingly pull the red further up, but also create a gradient between them.

In the image here you can see the red circles area has a smooth gradient, while the bottom of my mesh is a sharp edge where its fully red.

---

Okay so now the outside is painted red/black.
Time to do the exact same for the inside, but Green/Black.




Now that you colored both the inside and outside, you can unhide both layers. Your end result should look yellow-ish, because that's this special shader doing it.



Now we gotta merge the meshes together, otherwise the shader won't work in FO4.
Select both on the right hand side, right click and merge.

Save this NIF once more. Almost done!

Open this NIF in NifSkope one last time.
It should be apparent that the coloring worked.
Inside is green, outside is red, overlaps are yellow.
Different colors will mix differently.


In Step 16 we set the scale to 0.1, or whatever.
Change the scale back to 1.

Then save the NIF for the last time.
Close all instances of NifSkope and Outfit Studio if you have any left.
Delete any temporary or leftover NIF's. We're done touching the NIF files now.

We're going back into Creation Kit now.

Open your settlement in Creation Kit.
Search for "WorkshopBorder"

Select any of the borders and edit it.

Swap the model out. It'll ask you to change the FormID, click yes.
Save this new object.

Remember what prop you had selected last when making the Static Collection?
Right click that prop and duplicate it.


It will automatically select the Dupe you just made. So press Ctrl+F and swap that one for the new border object you made


It may be rotated 90°. Rotate it into place using the edit window.
Since the center is still the same as the item you replaced, you can just rotate it and call it a day.




Link it to the workbench using WorkshopLinkedBuildAreaEdge as Keyword.
Make sure "Initially Disabled" is selected.

Save the mod, launch the game, look at your border.


That's it. You did a thing!

If you need help with modding, join the Nexus Modding Discord[discord.com] server, I guess.
Although they could not help me when I had to figure out how to do this.
So I had to learn it via trial and error and wrote it down cause there was no guide for this.
Well, there was one guide, but they used Outfit Studio to guess the model shape, which is silly.

Anyhow. Good luck modeling this thang.

---

My Discord: https://discord.gg/5eDA7CE
I also got Paypal[paypal.me] for don't-ations.