garrettmills/commafeed
1
0
Fork 0
forked from Archives/Athou_commafeed
Code Pull Requests Activity

Fix off-by-one bug in HttpGetter.toByteArray()

Browse Source 
What this PR does:
Fixes an off-by-one error in HttpGetter.toByteArray where responses of exactly maxBytes were being rejected.
Now allows responses up to maxBytes, and only throws if the response is larger.

Why this change is needed:
Some feeds return responses exactly equal to the limit.
Current implementation incorrectly throws an exception, breaking parsing.

How it works:
Reads up to maxBytes + 1 bytes.
If actual size exceeds maxBytes, throws exception.
Otherwise returns full content.

Other notes:
No breaking API changes.
Safe for existing usage.
This commit is contained in:
AnasRazy /
2025-09-27 15:32:31 +05:30
committed by GitHub
parent d9e3cf0190
commit 65014d330a
1 changed files with 20 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
commafeed-server/src/main/java/com/commafeed/backend/HttpGetter.java
View File

@@ -242,26 +242,29 @@ public class HttpGetter {
return DateUtils.parseStandardDate(headerValue);
}
// ByteStreams.limit(input, maxBytes) reads at most maxBytes bytes.
// If the content length is exactly maxBytes, it throws an exception, even though the response is valid.
// This is an off-by-one error.
private static byte[] toByteArray(HttpEntity entity, long maxBytes) throws IOException {
if (entity.getContentLength() > maxBytes) {
throw new IOException(
"Response size (%s bytes) exceeds the maximum allowed size (%s bytes)".formatted(entity.getContentLength(), maxBytes));
}
try (InputStream input = entity.getContent()) {
if (input == null) {
return null;
}
byte[] bytes = ByteStreams.limit(input, maxBytes).readAllBytes();
if (bytes.length == maxBytes) {
throw new IOException("Response size exceeds the maximum allowed size (%s bytes)".formatted(maxBytes));
}
return bytes;
}
if (entity.getContentLength() > maxBytes) {
throw new IOException(
"Response size (%s bytes) exceeds the maximum allowed size (%s bytes)".formatted(entity.getContentLength(), maxBytes));
}
try (InputStream input = entity.getContent()) {
if (input == null) {
return null;
}
byte[] bytes = ByteStreams.limit(input, maxBytes + 1).readAllBytes(); // read one extra to detect overflow
if (bytes.length > maxBytes) {
throw new IOException("Response size exceeds the maximum allowed size (%s bytes)".formatted(maxBytes));
}
return bytes;
}
}
private PoolingHttpClientConnectionManager newConnectionManager(CommaFeedConfiguration config) {
SSLFactory sslFactory = SSLFactory.builder().withUnsafeTrustMaterial().withUnsafeHostnameVerifier().build();