Read the original article: Decouple your ShiftLeft AppSec policies with Open Policy Agent
The inspiration for this blog came from my recent preparation for an office hour on ShiftLeft Build Rules and Policy Language. Please note that this blog is based on my personal experimentation and doesn’t represent any official roadmap/direction of the ShiftLeft platform.
ShiftLeft code analysis platform offers a number of tools to help with your DevSecOps workflow. My recent favourites are the Build Rules, and Policy Language. With ShiftLeft Build Rules, developers could define a logic for breaking the CI/CD builds and prevent non-compliant pull requests from getting merged to the main branch using a YAML configuration file.
With ShiftLeft Policies, security engineers and champions could customize all aspects of the application security analysis — from performing selective scanning to customizing the sensitive data dictionary to even defining new business logic flaws — for either a specific application or for all applications in their organization.
More power to the developers and DevOps
Since policies could be used to influence the security findings across applications we limit this feature to only ShiftLeft administrators leaving build rules as the de facto tool for developers and DevOps folks.
As we move to a cloud-native world where the trio of concerns — application, infrastructure and security — are shifted to the left and engineered into the product, the YAML based approach used for the build rules feature has started to show its limitations.
Below are some recent enhancement requests from users of build rules:
- How do I specify a different build rule depending on the branch or the deployment environment? A pull request to the release branch could be treated different to a pull request made to another feature branch.
- How do I break the builds based on the freshness of findings? Example, to break builds only based on findings that are at least “x” days old.
- Can I programmatically exclude or include findings while calculating the total? Say based on some known source or sink methods.
One approach could be to enhance the build rules functionality to add more features — after all YAML supports any arbitrary key-value pairs, so we could technically add environment, findings_min_age_seconds and so forth for each customer request. Obviously, this is a recipe for building a kitchen-sink product.
A better approach could be to decouple policy enforcement away from the AppSec tool to a more general purpose policy engine. This engine could integrate with ShiftLeft platform via the API and interface with the CI/CD agent as a decisioning tool. This is where I believe Open Policy Agent comes in.
Fun with OPA
To fully understand OPA requires the reader to try the policies and rules hands-on. You should be a ShiftLeft user with a sample app and be willing to dedicate at least a couple of hours to proceed further.
opa is packaged as a single executable file thanks to go. opa policies are defined in a declarative language called Rego. There is initially a learning curve, but rego is definitely better than using jq and simpler than using bash or python while working with ShiftLeft API.
Let’s start our tutorial by setting the ShiftLeft Org Id and Access Token as environment variables. You can find these under Account Settings.
export SHIFTLEFT_ORG_ID=org id
export SHIFTLEFT_ACCESS_TOKEN=long token
We need some findings data to interactively try opa. You can use curl command to download the findings data for your app. For example, to download the data for shiftleft-java-example:
curl -H "Authorization: Bearer ${SHIFTLEFT_ACCESS_TOKEN}" -L "https://www.shiftleft.io/api/v4/orgs/${SHIFTLEFT_ORG_ID}/apps/shiftleft-java-example/findings?per_page=249&type=secret&type=vuln&type=extscan" -o "ngsast-raw-findings.json"
Download opa
curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_darwin_amd64
chmod +x ./opa
Check if there are any findings
Let’s begin by checking if there are any findings found in our ngsast-raw-findings.json. Save the below snippet as shiftleft.rego and use opa eval command.
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
To use the eval command:
opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.has_findings"
You should get true as the output if everything worked. The raw json was passed as input which can be accessed in the policy as a variable called input. has_findings is a complete rule which will be set to true if the built-in function count returns true. If there were no findings then has_findings would be set to undefined. By passing — fail-defined we ask the eval command to only fail (with exit code 1) if has_findings becomes defined.
Check if there are any critical findings
Now that we had some exposure to rego, let’s improve our policy to filter and look for critical findings. We add a new complete rule called `has_critical_findings` that will be set to true if there are findings with severity set to `critical`. (Other possible values are moderate and info).
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
has_critical_findings = true {
findings[_].severity == "critical"
}
$ opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.has_critical_findings"
Check if there are any critical or moderate findings
Looking for critical findings was easy. Now let’s implement a rule to look for critical or moderate findings which means we need to know the syntax for OR condition. With rego, to implement OR condition, you can implement the same rule multiple times! So, a rule called `has_non_info_findings` can be added once for critical and once for moderate. Yes, this is weird, but this is a deliberate design choice made by OPA developers.
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
has_critical_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "moderate"
}
$ opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.has_non_info_findings"
List the critical findings
Check the return code for 0 or 1 to break the build was cool. But a summary of findings matching a condition would be nice. Rego supports Incremental Rules which can be used to create a set or map of values. In the below example, `list_critical_findings` is an incremental rule that creates a map of key-value pairs.
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
has_critical_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "moderate"
}
list_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.severity == "critical"
}
$ opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.list_critical_findings"
List only critical XSS findings
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
has_critical_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "moderate"
}
list_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.severity == "critical"
}
list_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.severity == "critical"
}
list_xss_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.category == "XSS"
finding.severity == "critical"
}
opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.list_xss_critical_findings[id]"
List only XSS or Deserialization findings
By declaring multiple incremental rules with the same name `list_app_critical_findings` we can implement OR condition.
package shiftleft
default allow = true
scan := input.response.scan
findings := input.response.findings
has_findings = true {
count(findings) > 0
}
has_critical_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "critical"
}
has_non_info_findings = true {
findings[_].severity == "moderate"
}
list_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.severity == "critical"
}
list_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.severity == "critical"
}
list_app_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.category == "XSS"
finding.severity == "critical"
}
list_app_critical_findings[finding.id] = finding.title {
finding := findings[_]
finding.category == "Deserialization"
}
Followed by the eval command:
opa eval --input ngsast-raw-findings.json --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.list_app_critical_findings[id]"
Connect to the ShiftLeft API
Now that we have become an expert in rego, let’s make API calls and work with JWT directly from our policy.
For starters, let’s retrieve the org id directly from our ShiftLeft access token.
runtime := opa.runtime()
sl_access_token := runtime.env.SHIFTLEFT_ACCESS_TOKEN
payload := io.jwt.decode(sl_access_token)
sl_org_id := payload[1].orgID
To make an API request, we could use `http.send` built-in function. The complete rego script would look like this.
https://medium.com/media/5bf563b9b806e00fe7c06022b6461802/href
SHIFTLEFT_APP=shiftleft-java-example opa eval --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.list_app_critical_findings[id]"
OPA in server mode
OPA can also be run in server mode. This is useful if you want to run your policy server and setup gatekeeper-based policy controller for your kubernetes cluster.
SHIFTLEFT_APP=shiftleft-java-example ./opa run --server shiftleft.rego
$ curl localhost:8181/v1/data/shiftleft/allow -H 'Content-Type: application/json'
$ curl localhost:8181/v1/data/shiftleft/list_critical_findings -H 'Content-Type: application/json'
Branch specific rules
The last use case we are going to try is to implement branch-specific rules.
We can set a variable called branch based on the environment variable `GITHUB_REF` which would be set to the branch or pull request name on GitHub actions.
default branch = "master"
branch = "pr" {
contains(runtime.env.GITHUB_REF, "pull")
}
branch = "release" {
contains(runtime.env.GITHUB_REF, "release")
}
We can then implement allow rule based on this variable.
https://medium.com/media/7c169442ff6428051a8ac793dafbb22f/href
SHIFTLEFT_APP=shiftleft-java-example opa eval --data shiftleft.rego --format pretty --fail-defined "data.shiftleft.has_branch_findings"
Closing thoughts
Using OPA with ShiftLeft API thus opens up unique possibilities — an interesting build rules alternative, integration with Kubernetes admission controllers via gatekeeper. But most importantly using one generic policy engine to wire up and connect multiple security tools from various vendors is great for ensuring the success of your DevSecOps or cybersecurity initiative.
Have any questions or comments about OPA or ShiftLeft? Please feel to share with me on Twitter.
Become a supporter of IT Security News and help us remove the ads.