Bad News, Good News

Black Mesa Archive News & Developer Updates
#1

[align=center][/align]
Straight to the bad news: we are pushing back the release of Xen. We are truly sorry for getting everyone’s hopes up and then delaying… again. We worked very hard to make December, but we are not yet ready. As a team, we take FULL responsibility for that. We have an internal deadline we are confident in, and we will be getting everyone more details as we get closer to that date.

Thank you again to our community and Early Access supporters. The funding from Early Access has allowed us to hire many new talented developers, and has allowed older developers to put more time in the project. Simply put, Xen has proven to be an enormous undertaking, and while we are managing it to the best of our ability, it is proving to take longer than we estimated.

The good news is that we will be releasing a large update in December to test all of the new features that have been developed for Xen. We have changed a number of things within the Source Engine, and we want to make sure they all work on the myriad of PCs out there. By enabling the new tech on the Earthbound section of the game, we can get the engine/code fully stabilized on Steam before we drop the final chapters.

In December we will also have an update on where our Xen chapters stand, show some additional media, and deploy the engine update.

Here are some features that will be in the update:

Lens Flare
We’ve created a custom lens flare entity, and have added lens flares to important light sources in the game. Don’t worry, we will spare your eyes and not go overboard with our new lens flare tech. We think it adds a nice layer of polish to the Earthbound lighting and we hope to use it to really sell the alien feel of Xen… and add endless frustration to the debate about whether or not Gordon wears a helmet.

Dynamic Lights
We’ve shown some dynamic lights in the last update. Here are a few other sections that got the dynamic light treatment. Lights can simply cast light, cast shadows, and even cast volumetric godrays.

Security Joop with Dynamic Lights and Lens Flares

OAR Rocket Launch with Dynamic Shadows, Light Rays, and a Lens Flare

4 Way Texture Blends
We incorporated a 4 way texture blending system like the one seen in Counter Strike: Global Offensive, and improved upon it. Our level designers have 4 texture layers to blend, each with its own bump map and environment map (SSBM OR Normal Bump). Like other versions of the shader, level designers and artists can blend between the 4 layers procedurally using light and dark values of the diffuse textures.

Image Based Ambient Lighting
Lighting can now be generated from the image of the skybox texture. This gives level designers a very quick and accurate base lighting for outdoor maps. The example below has just a light_env in the scene, with no colors. All colored lighting is created automatically from the skybox image.

CSM and Godray Performance
We’ve improved the performance on Cascade Shadow Maps and Godrays and made them both more straightforward to implement on the Hammer side. Godrays are now placed with our newLight_Dir entity and automatically get ray angles from the light_env. Rays not only run better, but look smoother as well.

Environment Light Godrays

Rays From New Dynamic Lights

Xen Concept Art
Concepts as seen in PCGamer. In addition to the Xen screenshots we have released, these Interloper concepts highlight the difference in environment types we have.


Black Mesa 60% Off
We are also on sale until November 28th for the Steam Thanksgiving sale!

Stay Tuned
We will have a bunch more information next month!

#2

Just gimme the updated editor thingies and fixup the workshop tool and I’m ok with Xen being delayed :wink: .

#3

Gorgeous :slight_smile:

#4

As some one who is pretty much glued to a computer. I can wait. and man the new things look epic. though the lens flares I will never understand in games. it makes sense when looking at the sun but seriously where does that happen out side of that.

#5

As someone who wears glasses constantly, always
every time I look at a light
I’m not even sure if my life is real anymore

Also damn I love those dynamic lights

#6

Take yer time lads. And know that for the next 3 months there won’t be a complaint from me since we’ll also get an Update in December!

Cheers and take it easy. But keep us updated every quarter.

#7

Valve Time hits everyone :wink: But take all the time you need, it will be worth it.

Nice to hear you want to keep lens flare to a minimum. Personally I think all situations shown here look very good, except for the rocket launch. The flare is a bit too wide horizontally and I don’t think the volumetric light effects shouldn’t be that visible in that situation, since the rocket exhaust is such a wide light source. The sharpness of the god rays makes it look like it is a point light source. Or maybe it’s just a side effect caused by the GIF compression :wink: The dynamic lighting on the surrounding area looks great!

Nice to see the variety of areas Xen will have. As a Source hobbyist, I will probably spend a lot of time wondering how things were done…

And most importantly: that last GIF. Real-time day and night cycles? 8o

Really like the update. Keep up the good work!

#8

Thank you for the update!

#9

This is a crosspost from the Steam discussion forum (https://steamcommunity.com/app/362890/discussions/0/1489992080509804563/?ctp=5)

Thank-you everyone for your feedback and opinions on this issue, and the team most definitely does see this as an issue. I’ve seen some serious misinformation being bandied about in some of the comments, some of which seem antagonistic in nature, I will let them be for the time being, but would greatly appreciate it if opinions were not stated as fact in order to attempt to rile up bad feeling.

That being said I am acutely aware of the negativity and disappointment surrounding this delay, and whilst I wont give any further dates, I can only say we DO have an internal deadline and yes, we are renowned for being overly optimistic when it comes to developing on the source engine and estimating release dates.

I’d also like to thank Chon for fielding so many answers up til this point. My role on the team is Lead Designer, I’ve been with BM since mid 2006, so this has been a very long term project with many learning points. Anytime we have a situation like this, its disappointing.

The first time was in 2009, which was our ONLY other actual release date we attempted. The mod was released in 2012, but it wasn’t released due to a leak at all. That situation was a long term ongoing issue that we had to handle carefully internally in order to prevent a leak. The leaked video only had one impact, that of a single music track being changed, however any leaks of any nature are always disappointing, the only leak we’ve had was Alpha 6, which was early 2006 and was the main reason I was brought onto the team, where we literally redid the entire game from scratch as a result of it.

My role since mid 2014 was to design the entire baseline design for the Xen levels, and yes, we started the Xen levels in 2014 not because we hadn’t already looked at various concepts, builds and theories, but because after we got the news that the game would become EA, we decided to push for something more. This included essentially binning the old design ideas we had for Xen, and coming up with something more cohesive to Valves original visions, much of which i should point out were cut and not included.

The engine upgrades which some have stated are the reason behind the delays, are nothing of the sort. A lot of these things, such as lens flares, took perhaps a week to implement.
The only major thing we have spent a lot of time implementing, was Cascade Shadow Mapping. It’s important to remind, that we didn’t get the base code for this from Valve, and even had we gotten CSGO as our codebase, the CSM that is included with that, isn’t suitable for single player maps. As a result we implemented a very basic form of CSM (I’m sure Chetan our CSM guru will go into more detail at some point), which ignored light-styles (the problem with CSGO CSM). This unfortunately, along with the old god ray system, wasn’t well optimised and is one of the causes of reduced fps. Since the last update, this is something we have been working on to improve, that and the various implementation tests of deferred lighting and other technology.

We haven’t been working on this purely to try and make the game look as good as it can, but primarily to improve how it runs on peoples systems. CSM was one of the imperative features we needed due to the way our Xen levels are constructed, which is quite a different method from traditional additive mapping in source. As alot of the details are static props that are constructed and culled in Wallworm which is an integration tool between Hammer and 3dsMax, old static lighting, even with the most up to date staticproplighting settings on a per vertex basis, wouldn’t light all the rocks and unusual shapes correctly, whereas CSM does, in combination with that, our coders have looked into ways to better blend static lighting and prop lighting with the CSM, to keep the natural bounce of the static lighting method (which deferred lighting doesn’t do). All this has required a fair amount of testing and polish to get right.

At all times this testing and development has gone on, we have been working on iterating the game play and design of the Xen levels, starting with the base plans and working through them. We have found that Alien levels are remarkably hard to both concept and construct successfully. Some areas have been iterated upon multiple times to get the structure of the game play right, and some areas we’re still not happy with yet.

I’m painfully aware that a lot of the development cycle details are immaterial to you as a customer base, but we as a team are extremely focused on trying to get Xen to be as good as we can manage it. Like valve, we have had several situations where we’ve redone whole areas, cut whole areas. So far, we’ve not had to remake the entire game from scratch since 2006 (and we really have no intention of doing that). Having now had access to a lot of the original beta and unreleased test maps for HL1, including the unseen Xen maps, its quite shocking at how much they cut out. We’ve tried our best to include everything from the original Xen, whilst adding our own structure and flow, cohesiveness to the designs, in the same way we redesigned some of the earthbound sections, we’ve added back in some of the original cut areas that work with the overall structure of Xen and extended the game play significantly, to try and make this a worthy ending to the game.

I’ll keep an eye on this thread, if you have any questions, I will try to answer them (within the limits of our NDA of course :slight_smile: )

Again my apologies for the delay.

Chris Horn - Lead Designer

#10

Well I for one much prefer additional information, even if it’s information about a delay, to silence, although I imagine the general flaminess that follows every update makes silence a very appealing possibility.

I was always actually somewhat bothered by why the development team seemed to spend so much time implementing CSM (something that I as a level designer didn’t even find all that useful and was purely aesthetic) instead of making the underlying static lighting system more flexible so that I could finally evenly light a room that’s wider than it is tall. Learning that it’s for Xen actually clears up a lot of questions I’d had about the direction the engine development was going.

Do you know if basic lighting fixes like proper hard falloff are in the pipeline? Or is it all focused on the dynamic stuff at this point?

#11

We have a few improvements to the old vrad engine lighting, but primarily most of the tweaks we have are focused on the dynamic lighting. Most fixes to the static part of the lighting process are to fix issues with csm interaction with light styles. I think most of the coders by now have a large vrad shaped dent in their heads, from banging them against it for months at a time. But i’m sure one of our coders might be able to elaborate more.

#12

So we’re still dealing with a hybrid static-dynamic-csm lighting system, correct? I ask because I’m curious of the specifics as to how the dynamic lights interact with the different types of lighting, especially the baked static lighting.

1 Like
#13

This. Even in small maps with high-contrast lighting areas, it’s so hard to get all props to light up correctly. If you place 10 props in a dark area, you can expect half of them to be completely black or just lighted completely differently than the brush face under/beside it, all because VRAD uses different lighting methods as with brush faces. I can’t imagine how much of a nightmare it would be on a map with the amount of props Xen uses. If I recall correctly, you are also combining many props together which causes even more lighting problems, because VRAD sees them as one prop. So a completely different take on the lighting system is understandable and necessary to me. I think it was the right choice, and it probably has many other added benefits.

I think most of the complaints about the delays are coming primarily from people who have no experience with the Source engine. Yes, modern games can be developed much faster. Times have changed, engines have improved and the tools behind it have improved with them. Black Mesa’s engine has now been upgraded, but the tools didn’t.

Black Mesa really is something remarkable to me. I think anyone with the least amount of Source Engine experience can agree on this. I have a lot of respect for the whole team.

#14

With how much work you’re putting towards Xen, you could probably release it as a stand-alone game and be perfectly justified for doing so.

It would probably require too much work, but how feasible would it be to implement PBR into the engine and still hit a 2018 release? As I understand you’d have to go and pretty much refactor every single material, although I wonder if many of the assets were originally created with PBR materials and then “downgraded” for Source 1, so it wouldn’t be that much work then.

#15

Yeah, the lighting got more complex with the newlights Chetan developed, as we now have static lights, dynamic lights, CSM (for outdoor areas). Static lights would include point lights and spotlights, we use texture lighting, but in reality this is alot of point lights on the surface of the texture (we have some code that allows this to be better blended in now, so less blobbyness where the texturelights hit walls). The new lights are deferred dynamic lights iirc, the same type used in UE4 etc, as such the values we use are a bit different than normal. For example what would be a light with 255 255 255 200 settings, would need a brightness of 2000 with the new dlights to get roughly the same, but that often needs tweaking in game via the console based dlight editor we have setup. It can be a bit tricky to balance the new dlights with old static lighting, depending on the area you use them in, very dark areas can get overblown if you use too bright a setting. Dynamic lights don’t bounce either, bounce lighting is what gives source its realistic look, but deferred lighting has none, so you dont get a realistic look with it, unless combined. You also have to take into account the specular strength of surfaces, so shiny wet concrete vs dark matte textures can look quite different and needs to be calculated carefully. It’s quite a new system we put in, so alot of things are still being tweaked. You can have 48 of any dlight LOD (256, 512, 1024) in any map, but as each point light takes up 6 (as they act like a cube) and each spot takes 1 (single directional), you can quite easily overuse strong shadowmaps in an area, tanking fps.

A more technical overview of them would be better given by our coder, i just built the test maps and bugged him about features we wanted for them (the most recent being the ability to fade in and fade out light intensity/god ray intensity over time using a settings entity) :stuck_out_tongue:

#16

Do the d-lights combine for identical lightstyles like conventional named lights do? Where is their data stored- entity lump, lightmap lump, elsewhere?

#17

Thanks for reposting the initial update for members who don’t check the steam forums, and for crossposting the original info - There we so many negative comments on Steam that I gave up trying to find anything actually useful, and thus missed those posts.
As always, take all the time you need to ensure the final product is up to snuff - especially if it’s going to include bonus content based off cut HL1 maps! As for what is coming for us this year in the form of engine updates, I’ll do my utmost to make use of it all in the map I’m working on. :slight_smile:

I imagine a lot of the newer props are being textured in Substance or Quixel to take advantage of the quick, parameter-driven workflow they have (and to have some sweet renders to toss on their portfolios!) but a lot of the older assets that have been around for a while, as well as the vast majority of world textures, would probably have to be completely redone to work with PBR as they were made before the technique existed, let alone became standard. There are ways to re-work assets and make the specular map into the gloss/smoothness or metalic/roughness maps needed, but then you also have the issue of lighting detail painted into the diffuse texture, which, unless it’s all stored on separate layers and the source files are hanging around, would cause issues - since PBR is primarily a lighting method, having additional, less accurate lighting information in the texture would make it look kind of hokey.

I’d be super pleased to hear otherwise from an actual BM dev, but I kind of doubt we’ll be seeing a physically-based shading model in BM anytime soon, if ever.

#18

Chetan will be the man for answering this question, but I can tell you based on my limited knowledge of working with the lights. They’re nothing like conventional lights and it’s probably best to forget everything you know when it comes to using the new dlights. Apply UE4 knowledge rather than Source knowledge for these. I believe they are considered entities.

I may be summarily executed for sharing this information, so if you never see me again, you’ll know why. Chetan actually did add a PBR shader to our engine quite a long time ago (I think it was even before we had CSM) and I believe it was working. A lot of our props these days are made via PBR and then brought into Source and tweaked substantially from there (don’t quote me on that, I’m not an artist, I’m just going off vague memory). We found however, due to the fact that none of the game’s previously existing props were made via PBR, that there was basically no advantage to using PBR. All it did was hurt consistency with how we worked with props and how the props looked in game. So I think we stripped it out to reduce shader compile times (they are horrendous, it takes something like a day to compile the game’s shaders), and because nobody was using it anyway.

Disclaimer: All of this is from vague memory and I may be 100% wrong. Take everything I said here with a whole tub of salt.

#19

Hello guys, There were a lot of tech questions being asked so I am here to provide few more details.

First of all, let me clear this up that we are not delaying Xen just because we wanted to add some shiny stuff in the game. There were a lot of reasons for the delays as explained by Chon and Chris.
On the tech side alot of upgrades have been made in the engine of varying levels of difficulties and importance, For example new features like lens flare which didn’t took a lot of time to develop neither blocked Level Designers in anyway but there were these OTHER features both gfx & non-gfx (details will be out soon with December update) which were very limiting (if not blockers) for Level Designers to achieve acceptable results or to do things the way we wanted.

CSM -
CSM is one such feature which was very important for Xen, and consumed a fair amount of development time. it went through 3 upgrade cycles or major changes to improve quality, performance, to remove limitations from version 1 (shipped with cu3) and of course to fix new bugs. It’s still not finalized and we will keep improving upon it as needed and based on the feedback from December update. Among other things CSM now supports blending for lightmapped for better shadow color.

Deferred Lights -
This is sort of a new renderer on top of source’s renderer, so everything (well almost everything since we have to nuke some shader combos etc for perf etc) from old source will work as before and new lights will work on top of that.
So we now have -

  • Old Vrad Lights
  • Old Dynamic lights or D Lights with light styles etc
  • NewLight Entities which work with the deferred lights and shadow system. (Let’s call them DeferredLights or def-Lights to distinguish them from old dlights)

NewLight Entities are just source entities which register themselves to client side NewRenderer. Vrad doesn’t know or care about these entities and there’s no pre-calculation of any sorts for these lights.Also, there’s no GI or global illumination from these lights but it can be hacked via ambient light property
which allows you to add some illumination to shadowed regions instead of making them pitch black. There’s nothing special in stored in vbsp other than just a usual source entity data.
These lights are more like lights in Unreal/Unity than source lights. They can be static/stationary/dynamic and they have physically based inverse square falloff and intensity properties.
These lights do support light styles and can be parented, enabled/disabled etc.
We have also added a (sort of ) light editor in game so we can edit/config light in game itself then place them accordingly in hammer to avoid map recompiling everytime we need to change light color/range etc

NewPostProcess -
We support god rays for both sun and local point lights now. And we can lens flares to both local lights and sun. Godrays should run/look a lot better than our first version in cu3.
Postponing Effects are all optional. Everything can be disabled/enabled/adjusted via convars. We will never force anything on our customers and that’s just one big benefit of being a PC gamer we always get our gazillion gfx options to play with and customize our experience as we like.

VRAD
Upgrades have been made to Vrad as well like image-based lighting (sort of ) for props. Most of these upgrades have been implemented by Aleks (another programmer on the team). More details on VRAD upgrades in December.

PBR -
PBR is not a single effect or feature but a combination of a lot of features & effect underneath. I am afraid we wont be able to go for PBR even for Xen because of dx9, legacy/EB materials, and source. That being said we are borrowing some elements of PBR and implementing other upgrades wherever we can to make lighting & materials better than before without further delaying the release of the game.
Before we can even start working on PBR we would have to upgrade to dx11 for proper image-based lighting, I don’t think we can fully implement the image based lighting as needed for PBR on dx9/source.

Note - A lot of upgrades have been made to many parts of the engine (both shiny & non-shiny) and details will be out soon with the December update.
We still haven’t finalized the engine yet and it is still undergoing changes on daily basis for xen. But anytime we reveal any information about limitation or a particular feature its all based on current state of the engine and things might change in the future.
There will be a lot more information with the December update and I will be writing a blog post about some of the new features (with tech details). We call this new system NewRenderer & NewPostprocess, I will take a deep dive in my blog post in near future for people who are interested.
You guys will be able to find all the new relevant convars suffixed by nr_ and np_ when the December update is released.

We are sorry for the delay but I am confident you guys will love what we are working on when it is released and hopefully everyone will understand why we delayed the xen release.

I hope I have answered all the questions raised in comments above. I will try to be more active on the forums/steam to answer any further questions

Cheers!

#20

Really liking this changelist, chetanjags! It makes me want to move my mod to your version of Source for all of these neat tricks xP

Speaking of which, how did the new engine tech and asset pipeline affect bringing in assets? In the 2012 mod version, some HL2 textures and models made the leap from the base engine, but I remember seeing other devs posting about texture upgrade efforts. What has to be added or changed to work under NewRenderer and NewPostProcess? Or did all base assets get completely replaced for this version of Source?

Founded in 2004, Leakfree.org became one of the first online communities dedicated to Valve’s Source engine development. It is more famously known for the formation of Black Mesa: Source under the 'Leakfree Modification Team' handle in September 2004.