Show HN: finetune LLMs via the Finetuning Hub
2 by rsaha7 | 0 comments on Hacker News.
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for popular tasks, i.e., classification and summarization. Moreover, I would like to compare them against each other. So far, I have benchmarked Flan-T5-Large, Falcon-7B and RedPajama and have found them to be very efficient in low-data situations, i.e., when there are very few annotated samples. Llama2-7B/13B and Writer’s Palmyra are in the pipeline. But there’s so many LLMs out there! In case this work interests you, would be great to join forces. GitHub repo attached — feedback is always welcome :) Happy hacking!
Monday, 4 September 2023
New Show Hacker News story: latest news
Show HN: Rapidgzip – Truly Parallel Gzip Decompression with 10 GB/s
2 by mxmlnkn | 0 comments on Hacker News.
I have posted a much earlier version of this over a year ago [0]. Since then a lot has changed. Obviously, the name has changed. This happened for the paper publication [1]. I have also optimized the speed, integrated ISA-L for special cases, limited the compression-ratio-dependent maximum memory consumption, and finally added parallelized CRC32 computation, which adds ~5% overhead no matter the number of cores used. At this point, I am leaning towards calling it production-ready although there are still many ideas for improvements. Redoing the benchmarks of the older Show HN, would look like this: time pigz -d -c 4GiB-base64.gz | wc -c # real ~13.4 s -> ~320 MB/s time rapidgzip -d -c 4GiB-base64.gz | wc -c # real ~1.26 s -> ~3.4 GB/s However, at this point, the piping itself becomes a problem. Rapidgzip is actually slightly faster than cat when comparing the piped bandwidth! E.g., compare these additional benchmarks: time cat 4GiB-base64.gz | wc -c # real ~1.06 s -> ~3.1 GB/s time fcat 4GiB-base64.gz | wc -c # real ~0.41 s -> ~8.0 GB/s time rapidgzip -o /dev/null -d 4GiB-base64.gz # real ~0.68 s -> ~6.5 GB/s fcat is an alternative cat implementation that uses vmsplice to speed up piping. According to the ReadMe it currently is broken, but it works fine on my system and piping it to md5sum yields consistent results [2]. So, at this point, I/O and actually also allocations have become a limiting factor and if you want full speed, you would have to interface with the rapidgzip library interface directly (in C++ or via the Python bindings) and process the decompressed data in memory. The project ReadMe contains further benchmarks with Silesia and FASTQ data and scaling up to 128 cores, for which rapidgzip achieves 12 GB/s for Silesia and 24 GB/s when an index has been created with --export-index and is used with --import-index. It can also be tested with ratarmount 0.14.0, which now uses rapidgzip as a backend by default for .gz and .tar.gz files [3]. [0] https://ift.tt/2J7C31R [1] https://ift.tt/onAPDEf [2] https://ift.tt/oAeVB9s [3] https://ift.tt/80laHLT
2 by mxmlnkn | 0 comments on Hacker News.
I have posted a much earlier version of this over a year ago [0]. Since then a lot has changed. Obviously, the name has changed. This happened for the paper publication [1]. I have also optimized the speed, integrated ISA-L for special cases, limited the compression-ratio-dependent maximum memory consumption, and finally added parallelized CRC32 computation, which adds ~5% overhead no matter the number of cores used. At this point, I am leaning towards calling it production-ready although there are still many ideas for improvements. Redoing the benchmarks of the older Show HN, would look like this: time pigz -d -c 4GiB-base64.gz | wc -c # real ~13.4 s -> ~320 MB/s time rapidgzip -d -c 4GiB-base64.gz | wc -c # real ~1.26 s -> ~3.4 GB/s However, at this point, the piping itself becomes a problem. Rapidgzip is actually slightly faster than cat when comparing the piped bandwidth! E.g., compare these additional benchmarks: time cat 4GiB-base64.gz | wc -c # real ~1.06 s -> ~3.1 GB/s time fcat 4GiB-base64.gz | wc -c # real ~0.41 s -> ~8.0 GB/s time rapidgzip -o /dev/null -d 4GiB-base64.gz # real ~0.68 s -> ~6.5 GB/s fcat is an alternative cat implementation that uses vmsplice to speed up piping. According to the ReadMe it currently is broken, but it works fine on my system and piping it to md5sum yields consistent results [2]. So, at this point, I/O and actually also allocations have become a limiting factor and if you want full speed, you would have to interface with the rapidgzip library interface directly (in C++ or via the Python bindings) and process the decompressed data in memory. The project ReadMe contains further benchmarks with Silesia and FASTQ data and scaling up to 128 cores, for which rapidgzip achieves 12 GB/s for Silesia and 24 GB/s when an index has been created with --export-index and is used with --import-index. It can also be tested with ratarmount 0.14.0, which now uses rapidgzip as a backend by default for .gz and .tar.gz files [3]. [0] https://ift.tt/2J7C31R [1] https://ift.tt/onAPDEf [2] https://ift.tt/oAeVB9s [3] https://ift.tt/80laHLT
Sunday, 3 September 2023
New Show Hacker News story: latest news
Show HN: The simplest way to validate your idea
2 by Oxidome | 0 comments on Hacker News.
Real idea (The Facts Clock): https://ift.tt/CFkI2sj
2 by Oxidome | 0 comments on Hacker News.
Real idea (The Facts Clock): https://ift.tt/CFkI2sj
New Show Hacker News story: latest news
Show HN: FeedsBot – A Meower bot that posts RSS feed updates to groupchats
3 by mdwalters | 0 comments on Hacker News.
Hi, HN! I made this awhile back as a Meower[1] bot that posts RSS feed updates to groupchats, and reads RSS feeds, powered by Extractus' Feed Extractor[2][3]. My friend's groupchat uses this bot to post updates to his blog, and he says that it has been working out great. Looking forward to your feedback and suggestions! [1]: https://meower.org/ [2]: https://ift.tt/3aw9JhW [3]: https://ift.tt/G2mqrk8
3 by mdwalters | 0 comments on Hacker News.
Hi, HN! I made this awhile back as a Meower[1] bot that posts RSS feed updates to groupchats, and reads RSS feeds, powered by Extractus' Feed Extractor[2][3]. My friend's groupchat uses this bot to post updates to his blog, and he says that it has been working out great. Looking forward to your feedback and suggestions! [1]: https://meower.org/ [2]: https://ift.tt/3aw9JhW [3]: https://ift.tt/G2mqrk8
New Show Hacker News story: latest news
Show HN: Llama2.f90 – Toy LLaMA2 model inference in Fortran
2 by andy99 | 0 comments on Hacker News.
2 by andy99 | 0 comments on Hacker News.
New Show Hacker News story: latest news
Show HN: Command line query for all your unified log data
3 by tiwarinitish86 | 1 comments on Hacker News.
3 by tiwarinitish86 | 1 comments on Hacker News.
Subscribe to:
Posts (Atom)