You can embed JQL in a formula to check if the item (the one the formula is being calculated for) satisfies the condition of the query – that is, it will be a part of the query result.
The syntax is similar to calling an aggregate function:
IF JQL { assignee in membersOf("Team-Alpha") } : ...
The result of evaluating JQL {} is always 0 (false) or 1 (true).
Note that, unlike aggregate functions, these constructs do not use Expr, but rather another other languages, as the inner expression. Use the corresponding documentation as a reference for JQL.
The embedded queries are calculated separately from the Expr formula they are used in. Therefore, you cannot use any values from the Expr formula inside a JQL query or vice versa. Also, you cannot check the query match for any other item except the one the formula is being calculated for.
In other words, the only data that is passed between an embedded query and the outer Expr formula is 1 or 0 depending on whether the current item matches the query.