Unable to clone repo with LFS - getting error "Expected json type, got: text/html"
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Cloning a repository with LFS files from Bitbucket fails with the following error.
Error downloading object: /path/to/lfs/filename (<<hashid>>):
Smudge error: Error downloading /path/to/lfs/filename (<<hashid>>): batch response: Expected json type, got: "text/html"Environment
F5 Proxy or loadbalancer
Diagnosis
To diagnose the root cause, execute the below command on the client terminal to enable GIT verbose logging and then perform the GIT clone.
export GIT_TRACE_PACKET=1
export GIT_TRACE=1
export GIT_CURL_VERBOSE=1
export GIT_TRANSFER_TRACE=1
git clone <<clone-url>>Please refer the below Git clone verbose logs from cloning a test repository with LFS. From the verbose logging, we can see that the repository cloning completes first without any errors.
.....
remote: Compressing objects: 100% (13464/13464), done.
10:44:14.231870 pkt-line.c:80 packet: sideband< PACK ...
10:47:53.811615 pkt-line.c:80 packet: sideband< \2Total 28847 (delta 18320), reused 25704 (delta 15264)
remote: Total 28847 (delta 18320), reused 25704 (delta 15264)
10:47:53.814720 pkt-line.c:80 packet: sideband< 0000
Receiving objects: 100% (28847/28847), 532.40 MiB | 2.42 MiB/s, done.
Resolving deltas: 100% (18320/18320), done.
.....After successful download of repository file, when the git clone tries to download the LFS files from the Bitbucket it fails with below error.
10:48:03.944357 run-command.c:663 trace: run_command: 'git-lfs filter-process'
10:48:03.950323 pkt-line.c:80 packet: clone> git-filter-client
10:48:03.950356 pkt-line.c:80 packet: clone> version=2
10:48:03.950365 pkt-line.c:80 packet: clone> 0000
10:48:03.954212 trace git-lfs: exec: git 'version'
10:48:03.957239 trace git-lfs: exec: git 'config' '-l'
.....
<script type="text/javascript" src="/TSPD/08ee282cd3ab2000f1ef128bf315b63c3f9b8542d12293239defdbe43af82f4eac78738cc208d497?type=5"></script>
<noscript>Please enable JavaScript to view the page content.<br/>Your support ID is: 4945349409907659965.<link rel="stylesheet" href="/TSPD/?type=25" " /></noscript>
</head><body>
.....
Error downloading object: /path/to/lfs/filename (<<hashid>>):
Smudge error: Error downloading /path/to/lfs/filename (<<hashid>>): batch response: Expected json type, got: "text/html"
Errors logged to /path/to/clone/directory/.git/lfs/logs/yyyymmddT104808.500345156.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: /path/to/lfs/filename: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'Cause
The F5 proxy is built-in with various BOT Defence. If you've setup F5 as the reverse proxy from the Bitbucket Server or Datacenter application, during the Git clone the BOT falsely considers the LFS file downloads as a threat and blocks the file download to the client machine. F5 also automatically logs a Support Ticket to your F5 system as shown in the above logs("Your support ID is: 4945349409907659965").
Solution
This is a false alarm detected by the built-in BOT of the F5 proxy. Please speak to your dedicated F5 Support Engineer and make sure the <Bitbucket-Home>/shared/ directory is whitelisted from any scanning or denial by the F5 Proxy configuration.