I am no expert, just a small scale experimenter with under an acre of varied experiments
What to do about clay? To start, clay like this can be a gift. You have good clay for cobbing, adobe and other building projects. So scalping some selectively off the top is probably a good step. Harvest a bunch and put it in nice covered pile for other homestead projects.
Now to improve what you have I'd do the following:
1. Dig a few holes in areas you want to improve. 1 foot in depth. Analyze what you find. Is there clay for all 12 inches? Is it rock right under the clay? Is there anything better thereunder?
2. Assuming you have lots of depth of clay and rock and nothing extremely usable, your options are:
a. Rebuilding/building soil
b. Building raised beds or mounds
3. Rebuilding / building soil is always possible, but might not be economical depending on the size of area you want to rehab along with your time and budget.
4. Rebuild should consist of tilling into the loosened clay (recommend attempting to till/disc/etc.) the following:
a. Sand
b. Compost
c. potash or biochar
d. manure
You will need to amend this tilled area to a depth of 8 inches or more.
5. Consider planting rapid growing "weeds" that grow in your region. There are bunch of first to appear land rehabbers that do wonders and grow in really bad soil rather prolifically. Your local agriculture extension should be able to point you in the right direction. The idea with these "weeds" is many have sturdy and deep taproots. Chicory is one my favorites along with dandelion. Both incidentally are very edible and useful. Let these grow and chop and drop them in place when large enough, leaving their roots in the ground. Only pull by roots the noxious and useless weeds.
6. Trees and shrubs usually are your friend in rehabbing the land. They produce lots of canopy cover, provide animal habitat, leaves for soil enrichment, better water management in the soil and tilling effect underground. Find quick growers local to you, native trees and shrubs and plant lots of them. I am fond of edible trees and shrubs.
I'd do all of that along with reclaiming what I could from the gully.
The cost of carting in materials today is outrageous. Fuel surcharges are normal business today.
I'd be looking for a local arborist nearby who will give you all the chippings and woody cuts you want for fuel cost.
Nothing wrong with raised beds though
But depends on your intended land use there. For growing food, they are superior in many aspects.