The machine that runs CGRAN down in some basement somewhere at Carnegie
Mellon has hit some issues again. Given that I’m no longer at the
university, these issues are becoming harder for me to address. At this
point, it’s probably best for CGRAN to “move on” as we’ve all been in
discussion about over time.
What I can do if everyone still finds CGRAN useful is:
Provide a more reliable host and machine for it
Update it to be more useful to the community (e.g., more towards
git)
It still gets a lot of hits (~16,000 a month) and every time it goes
down
people hunt me down and ask when it’s coming back up. So it seems as
though the community still uses it.
I can update it with Pybombs or Gitlib or whatever people feel is
appropriate. It can be more of a portal page even, without a repository
if
most people just use Github now anyway. Do people still like it is a
standalone service, or is it better to just “roll it in” to the GNU
Radio
webpage somewhere now? I want to do whatever the community finds is
most
useful.
CGRAN is useful and I’d like to see it live on in some form, at least as
a
directory of modules that people are developing. My gut feeling is that
github is a good way forward.
I’m not sure what kind of quality assertion you want to make by
including
something in CGRAN (clueful developers? decent module architecture?
likely
to be reusable? unlikely to be abandon-/gradstudent-ware…), but having
a
curated directory seems better than searching for “gnuradio” and finding
527 stupid repositories of random grc experiments that only work with
GR3.5.
I found the GR3.6 to 3.7 migration page last night and I suddenly find
myself motivated to fork all these old modules and try update them to
the
new API. Anyone got any statistics on how community involvement changed
after their projects moved to or from github?
This is something that comes up at our dev calls (and other dev
meetings) regularly, and we really need to address it sooner rather than
later.
George, if the support burden is getting to much, we can surely fix a
short-term solution by migrating stuff to some temporary location (let’s
take this specific discussion offline, though).
In the future, we’d like to have something that ties in nicely with
Pybombs, and also uses the gits. How exactly, that’s something we need
to decide, and any community input on this is appreciated.
Marcus, thanks for keeping up-to-date projects on CGRAN! Since you’ve
always been actively involved, what would you like to see different
and/or
improved? It can still be the place where your projects live, I am just
trying to make CGRAN more friendly to changes in the current community
and
to be more supportive of newer projects. The clear example has always
been
making it more git-centric.
Thanks for the feedback, Chris and Martin. What I’m going to do is keep
CGRAN down until we have some sort of plan/resolution and use it as a
form
of motivation. Every time I’ve managed to resurrect CGRAN from the
dead, I
just leave it go and forget about it for some time again. I think that
the
down time might help us come to a conclusion sooner.
Several people have e-mailed me about access to the repository. I was
able
to get the repo back up, and it should be anonymously readable here: https://www.cgran.org/svn/projects
To address Chris’ thoughts, I’ve always felt CGRAN was useful in two
aspects: 1) To find useful and up-to-date projects (albeit rare), and
2)
To find more historical projects that highlight the capabilities of GNU
Radio and SDRs and to resurrect and/or build from them. I know the
latter
has been a killer, but I’ve found multiple times that people came to
CGRAN
to dig up old code and build something new from it. But if anything,
these
two types of projects need to be clearly marked and separated.
Academically, I know that students are very willing to take brutally
dead
code and use pieces of it for projects.
Maintenance over time is simply just difficult. Once projects are
complete, many people move on but GNU Radio keeps on chugging. I know
that
I lost time to maintain my projects. Pybombs could at least guide the
user
to get correct versions, let them know there is a mismatch, etc. It can
also provide the link from a project to where the actual code and
repository are. I think that pybomb entries can point to github
locations,
right?
When CGRAN was inaugurated, github wasn’t as popular as it was (and GNU
Radio was still on SVN itself). We would not have gone for a central SVN
repo if github had been on our radar back then.
I guess most people either share Marcus’ sentiment, or are biased
towards git. So, ditching SVN is pretty much a no-brainer.
However, one major difference between SVN and git is that the latter
doesn’t have the concept of every dir being a repo in and of itself.
This means if we simply pushed everything to a giant github repo, that
would not be terribly useful (definitely not a replacement for CGRAN),
although I can see that being a temp solution so that at the very least,
nothing is lost (a big advantage of using github is that they’re less
likely to lose data).
Really, every CGRAN project should be pulled into it’s own little repo,
e.g. on github. Migrating from SVN to git is really easy (even with
preserving history and all). I guess we could put up instructions on how
to do that if there’s popular demand.
However, there’s also the wiki pages on CGRAN. We do need a strategy for
those (and a way to access them).
I agree with Martin that once we go to git, every project has its own
independent repo. That shouldn’t take much time at all to do, I can
just
run some svn2git magic to spit out separate repositories. The question
will be where those repositories live. I can host the repositories
again.
I could replace the tired Trac interface with Gitlab and then host the
repositories locally and through there. If that’s the case, Github
repositories could be forked in Gitlab and/or point to the Github repos?
(e.g., for people who only want their code on Github). I think the
downside of Gitlab is that it doesn’t seem to be very customizable to,
for
example, have a coherent single Wiki of some sort like Trac dd. It will
be
a bunch of separate Wikis buried in to each separate repository’s page.
So I think we are agreeing so far on git with multiple repositories for
each project. What we need to figure out is what the frontend is.
Marcus, thanks for keeping up-to-date projects on CGRAN! Since you’ve
always been actively involved, what would you like to see different and/or
improved? It can still be the place where your projects live, I am just
trying to make CGRAN more friendly to changes in the current community and
to be more supportive of newer projects. The clear example has always been
making it more git-centric.
One of the things I find myself constantly doing for projects on cgran
is making manual snapshots. Github has the ability to make a snapshot
both for head and for any tags, which has always been very useful.
While I don’t care if no one wants to use github, or even git for that
matter, some form of automatic snapshot for tags and/or revs would be
very useful imho.
that seems to be a nice solution you’re proposing, George. What about
having a uber-repo that uses external submodules? This way, you could
have your single CGRAN repo, with all the packages as submodules, some
documentation in a single wiki, all per gitlab, and just keep the
projects as independent repos, hosted on a cgran machine or on
github/osmocom/wherever. We get the functionality to backup “all the GNU
Radio ecosystem” at once by running some git submodule update command,
and pybombs could just clone that repo, and init submodules as the user
installs them.
Marcus, thanks for keeping up-to-date projects on CGRAN! Since you’ve
always been actively involved, what would you like to see different
and/or improved? It can still be the place where your projects live,
I am just trying to make CGRAN more friendly to changes in the current
community and to be more supportive of newer projects. The clear
example has always been making it more git-centric.
I have no religious convictions about git vs svn.
I’d have to change a couple of scripts to pull from git if things were
migrated to git, but apart from that, it’s no big deal for me.
I haven’t actually used the web interface for CGRAN for quite some time.
So I don’t have much of a feel for what exciting new things need
to go with any re-design.
I definitely think this needs a wider discussion, and keeping the
existing repo alive is a good thing, since there are folks using
CGRAN-resident
Gr-based tools.
Marcus, I like the idea of an uber-repo with external submodules. That
would mean these submodules could link to a repo we could provide the
user,
a repo they already have on github, or a repo they have on some other
external server. But in the end, our uber repo would point to all of
them
and then they could update the commit that external submodule points to
over time. Thanks for that suggestion.
Rick, thanks for this suggestion also! I will make sure that we are
able
to include some sort of snapshot. When you say snapshot here, does that
act as some sort of release or history? It must be different than a
tag,
since you say tags are part of a snapshot. Can you give me an example
snapshot provided by some other service?
Marcus, I like the idea of an uber-repo with external submodules. That
would mean these submodules could link to a repo we could provide the
user, a repo they already have on github, or a repo they have on some
other external server. But in the end, our uber repo would point to all
of them and then they could update the commit that external submodule
points to over time. Thanks for that suggestion.
Sorry for being a buzzkill here,
but I don’t really like this except as a temporary/transition solution.
Assume CGRAN really takes off and grows. Do you really want all OOTs out
there in a single repo? What exactly is their logical connection, which
would warrant them all being tied together in a super-repo?
This would require someone to keep updating the submodules, too, which
seems unnecessary.
In the long run, I would assume people want to host their OOTs on github
(or similar services), and CGRAN would simply be a link to those.
As I said, during a transition time, we might want something else.
But submodules are messy, and I suggest not using them for this
particular application.
since you say tags are part of a snapshot. Can you give me an example
snapshot provided by some other service?
For example a github repo:
This link to a tag shows you the state of the repo when it was tagged,
but the exciting part is along the right hand side where it says
“download zip”:
Github also supports tgz (but not tbz2 or txz because apparently github
is from 1995).
To be able to download any commit as a tarball is a very very useful
feature on github. Again, I don’t actually care about git or github,
just that this specific feature is great. For some reason a lot of SDR
projects just never seem to make releases (sometimes not even tags) and
this saves me having to make manual tarballs (it also helps the source
tarball be the same across all distros since we don’t each have to make
our own).
but I don’t really like this except as a temporary/transition solution.
Assume CGRAN really takes off and grows. Do you really want all OOTs out
there in a single repo? What exactly is their logical connection, which
would warrant them all being tied together in a super-repo?
This would require someone to keep updating the submodules, too, which
seems unnecessary.
In the long run, I would assume people want to host their OOTs on github
(or similar services), and CGRAN would simply be a link to those.
As I said, during a transition time, we might want something else.
But submodules are messy, and I suggest not using them for this
particular application.
Since, git 1.8.2 a git submodule can track a branch [1] (as opposed to
pointing to a commit). That means that it is not necessary to update the
submodules in the super-repo. A submodule is very similar to a pybombs
recipe, a link to a repo and a branch.
Most likely most of the repos (i.e. GNU Radio apps/modules) will be
hosted at github or similar services. So nothing would change for the
authors of the modules. (That’s at least my understanding of the
proposed solution)
Having a directory of projects along with metadata such as author name
and contact, status, GR API version compatibility, etc along with a
source mirror and link to origin is really valuable. From this
perspective I like CCAN: they have a ‘package’ format so they can
parse and display that metadata in a convenient way on their web site
(for an example see ccan ). We can
use gr_modtool to either fill in a mod_info type file or request
authors put data in their README in a nice way. Then CGRAN can display
all of the project information, potentially provide a mirror, and
point to the original source.
I agree with Martin that the super-repo feels weird in the sense that
there is no shared history or relationship between the histories of
each project. If the goal is to make it easy to clone all projects at
once then pybombs almost does that already and it’s very easy to
modify (actually it’s probably a bug that pybombs fetch doesn’t work
with --all). I do think it is convenient/useful to have a single
base-URI to get all OOT modules from assuming someone doesn’t mind
paying for the bandwidth to have people constantly downloading OOT
projects. Also, what happens to a submodule if the origin disappears
(for example someone loses interest and eventually deletes their
github repo)?
Let me add one more thing about CGRAN while we are still trying to
narrow
down how to handle this. One reason I put CGRAN in place was to host
code
that might “disappear.” For example, students post it on university
webpages and when they graduate, that hosting gets taken down.
Additionally, someone creates an account on some code hosting service,
and
the project later gets taken down. That was the “archive” part of
cgrAn.
I guess the more we talk about just linking to a bunch of things, I
wonder
if we will lose one of those initial goals.
It looks like github can mirror another git repo. It looks like mirrors
don’t update automatically; the docs suggest using a cron job to
automate,
so if the hypothetical grad student’s repository goes away the CGRAN
version could remain at the last published version.
Interesting, Chris. Thanks for that info. The question would be, do we
mirror everything? It seems like that’s a possibility. It’s nothing I
think I could automatically set up. I typically have left it up to the
authors to decide given their software license.
New module authors could contact the CGRAN admin(s) to be listed in the
directory and if an admin finds a really interesting module they can
reach
out to the author to say “your project seems neat, could you send us a
request to be included in CGRAN?”.
A CGRAN admin would add them to the mirroring tool config (just a list
of
repo URLs), and the mirror tool could check the license status before
updating - possibly by ensuring that {LICENSE,COPYING}{,.txt,.md} exists
and contains some known text.
I’d be happy to help write some of the automation. I haven’t yet tested
what happens when I fork a repo and the upstream deletes their repo…
does
all my work go away?
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.