1 file listdir apache-spark-sql pyspark palantir-foundry
我正在寻找一个代码来获取 pyspark 上 palantir 铸造厂目录中的文件和文件夹名称列表。因此,如果我提供铸造路径位置,我需要使用 pyspark 代码的文件和文件夹名称列表。
有人可以帮我解决这个问题吗?
小智 5
我们在内部使用此函数来查询文件夹的子对象。请注意,客户端分页未实现:
def get_child_objects_of_folder(self, folder_rid: str, foundry_hostname: str, auth_token: str) -> list:
"""
Returns the child objects of a compass folder.
Args:
folder_rid: Compass folder rid,
e.g. ri.compass.main.folder.f549ae09-9534-44c7-967a-6c86b2339231
Returns: (list of dict's) information about child objects
"""
response = requests.get(
f"https://{foundry_hostname}/compass/api/folders/{folder_rid}/children",
headers={
"content-type": "application/json",
"authorization": f"Bearer {auth_token}",
},
)
response.raise_for_status()
response_as_json = response.json()
if (
"nextPageToken" in response_as_json
and response_as_json["nextPageToken"] is not None
):
raise ValueError(
f"{folder_rid} has too many children "
f"({len(response_as_json['values'])}), "
f"paging currently not implemented."
)
return response.json()["values"]
Run Code Online (Sandbox Code Playgroud)