Show HN: estela, a modern elastic web scraping cluster
2 by breno | 0 comments on Hacker News.
estela is an elastic web scraping cluster running on Kubernetes. It provides mechanisms to deploy, run and scale web scraping spiders via a REST API and a web interface. It is a modern alternative to the few OSS projects available for such needs, like scrapyd and gerapy. estela aims to help web scraping teams and individuals that are considering moving away from proprietary scraping clouds, or who are in the process of designing their on-premise scraping architecture, so as not to needlessly reinvent the wheel, and to benefit from the get-go from features such as built-in scalability and elasticity, among others. estela has been recently published as OSS under the MIT license: https://ift.tt/ax9sgHG More details about it can be found in the release blog post and the official documentation: https://ift.tt/DjdpEiJ https://ift.tt/4FrcJjf estela supports Scrapy spiders for the moment being, but additional frameworks/languages are on the roadmap. All kinds of feedback and contributions are welcome! Disclaimer: I'm part of the development team behind estela :-)
Wednesday, 7 September 2022
New Show Hacker News story: latest news
Show HN: WASM/TS crypto library for Ed25519, Shamir sharing, AEAD secret boxes
2 by fuzzc0re | 0 comments on Hacker News.
Hello HN, I made this library because I wanted better performance than tweetnacl and I also wanted Shamir secret sharing. I wrote some C functions that use libsodium and compiled them to WASM with emscripten and everything is fully typed. Would love to hear your thoughts on this. Try it out, and of course feedback and contributions are very welcome!
2 by fuzzc0re | 0 comments on Hacker News.
Hello HN, I made this library because I wanted better performance than tweetnacl and I also wanted Shamir secret sharing. I wrote some C functions that use libsodium and compiled them to WASM with emscripten and everything is fully typed. Would love to hear your thoughts on this. Try it out, and of course feedback and contributions are very welcome!
New Show Hacker News story: latest news
Show HN: Build SaaS application from scratch with React and Django
3 by pplonski86 | 1 comments on Hacker News.
3 by pplonski86 | 1 comments on Hacker News.
New Show Hacker News story: latest news
Show HN: Executing Python from Tweets
2 by guilatrova | 0 comments on Hacker News.
Hello HN! Pyrun is a Chrome Extension that brings a Python IDE into your Twitter for those who consume and create Python content. *The Status Quo (This is quite a limited experience!)* - Content creators share Python snippets as images; - The audience zooms into the image to see what’s in there; * The Pyrun extension creates a whole new learning experience:* ▶ Hit one button to execute the Python snippet See Python's output inside Twitter Edit the code and rerun Content creators only have to share the code inside the image’s ALT, add “#pyrun” in their Tweet and that’s it! This extension will enhance the learning experience and improve engagement without disrupting the Twitter experience!
2 by guilatrova | 0 comments on Hacker News.
Hello HN! Pyrun is a Chrome Extension that brings a Python IDE into your Twitter for those who consume and create Python content. *The Status Quo (This is quite a limited experience!)* - Content creators share Python snippets as images; - The audience zooms into the image to see what’s in there; * The Pyrun extension creates a whole new learning experience:* ▶ Hit one button to execute the Python snippet See Python's output inside Twitter Edit the code and rerun Content creators only have to share the code inside the image’s ALT, add “#pyrun” in their Tweet and that’s it! This extension will enhance the learning experience and improve engagement without disrupting the Twitter experience!
Tuesday, 6 September 2022
New Show Hacker News story: latest news
Show HN: VoxelChain – An Experimental Voxel Engine
3 by x1m4 | 0 comments on Hacker News.
VoxelChain is an experimental tool to create voxel worlds in the browser. The lighting is fully ray traced in real-time and there is powerful cellular-automata based programming system, which allows to create complex digital circuits and model the behaviour of voxels (behaviour such as falling sand or water). Technology wise, I'm using WebGL2 for the rendering and the simulation is coded in C89 and gets compiled to WebAssembly (with multi-threading) using Clang. The simulation is basically a custom cellular automaton and is fully parallelized. Once WebGPU is released, I'm planning to run the simulation on the GPU, instead of the CPU, which will give a massive speed up. I've been working on this project full-time for over a year now, and finally came to the point of realising a public version of it to play with. There is already some really cool stuff that the community has built, and it's super fun to see how everything evolves! Let me know what you think and feel free to ask any questions :>
3 by x1m4 | 0 comments on Hacker News.
VoxelChain is an experimental tool to create voxel worlds in the browser. The lighting is fully ray traced in real-time and there is powerful cellular-automata based programming system, which allows to create complex digital circuits and model the behaviour of voxels (behaviour such as falling sand or water). Technology wise, I'm using WebGL2 for the rendering and the simulation is coded in C89 and gets compiled to WebAssembly (with multi-threading) using Clang. The simulation is basically a custom cellular automaton and is fully parallelized. Once WebGPU is released, I'm planning to run the simulation on the GPU, instead of the CPU, which will give a massive speed up. I've been working on this project full-time for over a year now, and finally came to the point of realising a public version of it to play with. There is already some really cool stuff that the community has built, and it's super fun to see how everything evolves! Let me know what you think and feel free to ask any questions :>
New Show Hacker News story: latest news
Show HN: Simulate dollar-cost averaging in any mix of stocks
15 by dwmcc | 6 comments on Hacker News.
15 by dwmcc | 6 comments on Hacker News.
New Show Hacker News story: latest news
Show HN: I built an open source job scheduler to help script everything
4 by galdor | 0 comments on Hacker News.
As a software engineer, my reflex has always been to script everything I can. Scripting is easy, but everything around it is tricky: scripts have to run somewhere, they need access to credentials, they need to be monitored. None of it is simple. No-code automation platforms such as Zapier or IFTTT are too restrictive for technical tasks; CI/CD platforms make it easy to script tasks related to a repository, but are limited when trying to do anything else. I wanted the low level control you have when writing scripts while retaining the comfort of a platform managing execution for me. So I built a program for that. Eventline schedules, runs and monitors any script. Jobs can be executed on various runtimes (currently locally, with SSH, in Docker or in Kubernetes), manually or in reaction to various events. You then get observability (script output, history, execution stats…), full control (with the web UI, HTTP API or command line tool) and the ability to share scripts with others. Eventline 1.0 is out. I use it myself, so does a friend and a first client. The roadmap is packed with features which will make Eventline even more useful: global registry for value injection, sub-jobs, scratchpad to share data between jobs, artifacts, ACL (and SSO for Eventline Pro), a lot more connectors, identities and events… I quitted my job in 2020 and created a company, Exograd, hoping to become financially independent. This has been quite a ride, both exhilarating and scary: I made lots of mistakes and learned more in 22 months than in my entire career. I would like to bring enough revenue to be able to live from it, and then work on expanding Eventline and developing new technical products. I currently offer a support contract, a proprietary extension (Eventline Pro) for enterprise use, and cloud hosting to use Eventline without having to host it. Happy to answer any question!
4 by galdor | 0 comments on Hacker News.
As a software engineer, my reflex has always been to script everything I can. Scripting is easy, but everything around it is tricky: scripts have to run somewhere, they need access to credentials, they need to be monitored. None of it is simple. No-code automation platforms such as Zapier or IFTTT are too restrictive for technical tasks; CI/CD platforms make it easy to script tasks related to a repository, but are limited when trying to do anything else. I wanted the low level control you have when writing scripts while retaining the comfort of a platform managing execution for me. So I built a program for that. Eventline schedules, runs and monitors any script. Jobs can be executed on various runtimes (currently locally, with SSH, in Docker or in Kubernetes), manually or in reaction to various events. You then get observability (script output, history, execution stats…), full control (with the web UI, HTTP API or command line tool) and the ability to share scripts with others. Eventline 1.0 is out. I use it myself, so does a friend and a first client. The roadmap is packed with features which will make Eventline even more useful: global registry for value injection, sub-jobs, scratchpad to share data between jobs, artifacts, ACL (and SSO for Eventline Pro), a lot more connectors, identities and events… I quitted my job in 2020 and created a company, Exograd, hoping to become financially independent. This has been quite a ride, both exhilarating and scary: I made lots of mistakes and learned more in 22 months than in my entire career. I would like to bring enough revenue to be able to live from it, and then work on expanding Eventline and developing new technical products. I currently offer a support contract, a proprietary extension (Eventline Pro) for enterprise use, and cloud hosting to use Eventline without having to host it. Happy to answer any question!
Subscribe to:
Posts (Atom)